Function Retrieval: Add unlimited functions to ChatGPT
Ever since the launch of ChatGPTs Function calling abilities on June 13 2023, developers have been able to make ChatGPT decide between a list of available functions it should run in order to solve a task.
GPT-4 later got a new update with the 128k turbo model of November 6th. With this change, some syntax of the API also changed. The list of functions we pass to ChatGPT is now called tools. The reason for this is because OpenAI has added their own self hosted tools such as Code Interpreter and Retrieval for the Assistants API. You might already be familiar with these as they are baked into ChatGPT in the browser.
Function calling
This article is focused around the function side of tool calling in ChatGPT, operating within the Chat Completion API. To define custom functions we can do something like this:
tools = [
{
type: "function",
function: {
name: "add_numbers",
description: "Add two numbers together",
parameters: {
type: "object",
properties: {
number1: {
type: "number",
description: "The first number to add"
},
number2: {
type: "number",
description: "The second number to add"
}
},
required: ["number1", "number2"]
}
}
},
{
type: "function",
function: {
name: "subtract_numbers",
description: "Subtract number1 from number2",
parameters: {
type: "object",
properties: {
number1: {
type: "number",
description: "The first number to add"
},
number2: {
type: "number",
description: "The second number to add"
}
},
required: ["number1", "number2"]
}
}
},
]
history = [{"user":"Whats 4.313523449123 + 124.213024124?"}]
response = openai.chat.completions.create(
model='gpt-4-1106-preview',
messages=history,
tools=tools,
)In the example above we added two simple tool descriptions of type “function”. The response from ChatGPT will tell us to invoke the add_numbers() function, which we then would need to run on our end before we send the results to ChatGPT. We’ll get back to how this is done later.
One thing that is important to understand is that the list of tools (our functions) are counted towards the token usage. Imagine passing not only 2, but 20, or 200 functions to ChatGPT, every time. Even though the new GPT-4-turbo model can handle 128k tokens, and we are likely to see even bigger context windows going forward, we generally want to reduce the number of functions we give to ChatGPT for a few reasons:
- Cost — We pay for each token we use, and even though models will become cheaper over time, we need to think of these cost reductions in percentages. Reducing the length of a prompt with 90% is a cost reduction of 90% for the input tokens. As long as the price of LLM’s are not trivial, which they currently are not, we want to limit the text presented in the context window as much as possible, as long as it’s not affecting the quality of the output.
- Noise — When we reduce irrelevant information, we tend to get better answers. This means that adding a lot of unwanted functions in a prompt will only create noise for the model, which can make its decision making abilities worse, and thus give us answers that are not optimal.
The approach of just adding all our functions to the prompt looks something like this:
Here we provide the functions to GPT-4, which gives the model the possibility of choosing between all the functions.
Function Retrieval
To reduce the number of functions we pass to ChatGPT, we need to pass only the most relevant functions. To find the functions that are relevant for the prompt, we can use the same setup that we use for RAG (Retriever Augmented Generation). If you’re unfamiliar with RAG, it is a technique used for retrieving documents that are relevant to what the user asks. This is usually done by searching through a pre-defined vector database containing all our documents, find the most relevant documents, and add these documents to the context before we send it to the language model. This can also be done with functions, and by following this inspired technique, we can in theory add “unlimited” functions to ChatGPT.
In my extended example I tested with around 50 functions, including functions such as flipping a coin, multiplication, division, converting between metrics, telling random jokes and more. You’ll find the example here. Below is a question I asked to test:
”Flip a coin, then add 1 for heads, or 2 for tails to a random number from 1 to 10 and divide it by the number of the current day in the week before squaring the result and then convert it from Fahrenheit to celsius. If the number is greater than 10 do a daily horoscope for me, or else tell me a random joke.”
And here is the output combined with the Speech API🗣️
How to do Function Retrieval
1. Create a python project
Create a new python project and install the required packages.
pip install openai pinecone-client python-dotenvCreate an .env file. We’ll be using Pinecone for the vector database to store the functions. Make sure you include the Pinecone API Key alongside with your OpenAI API Key in the .env file.
PINECONE_API_KEY=<FILL_IN>
OPENAI_API_KEY=<FILL_IN>2. Create your functions
Even though we don’t pass with all our functions to ChatGPT, we need to define them on our end so that ChatGPT is able to invoke the functions when needed. For this example we use some simple functions, but an extended example of functions can be found here.
# Add two numbers together
def add_numbers(args):
return args["number1"] + args["number2"]
# Subtract one number from another
def subtract_numbers(args):
return args["number1"] - args["number2"]
# Multiply two numbers together
def multiply_numbers(args):
return args["number1"] * args["number2"]
# Add more functions here
function_map = {
"add_numbers": add_numbers,
"subtract_numbers": subtract_numbers,
"multiply_numbers": multiply_numbers,
# Map rest of the functions here
}The most tedious work when it comes to functions is to describe your functions in the correct way. In this example I have saved the descriptions in a JSON file “functions.json”, which will be loaded into our function db. To save some time, I created a GPT, which converts functions to the correct descriptions for you. Just put your python functions in, and it will convert it for you. If you’re looking for a bigger set of example functions to use, you can find a collection of around 50 function descriptions here, and the python functions mentioned above here.
[
{
"type": "function",
"function": {
"name": "add_numbers",
"description": "Add two numbers together",
"parameters": {
"type": "object",
"properties": {
"number1": {
"type": "number",
"description": "The first number to add"
},
"number2": {
"type": "number",
"description": "The second number to add"
}
},
"required": ["number1", "number2"]
}
}
},
{
"type": "function",
"function": {
"name": "subtract_numbers",
"description": "Subtract one number from another",
"parameters": {
"type": "object",
"properties": {
"number1": {
"type": "number",
"description": "The number to subtract from"
},
"number2": {
"type": "number",
"description": "The number to subtract"
}
},
"required": ["number1", "number2"]
}
}
},
{
"type": "function",
"function": {
"name": "multiply_numbers",
"description": "Multiply two numbers together",
"parameters": {
"type": "object",
"properties": {
"number1": {
"type": "number",
"description": "The first number to multiply"
},
"number2": {
"type": "number",
"description": "The second number to multiply"
}
},
"required": ["number1", "number2"]
}
}
}
]3. Vectorise your functions
In this example we’re using Pinecone as a Vector database for storing our functions, but you can easily do this with other providers as well.
import json
import os
import openai
from openai.types.chat import ChatCompletionMessageToolCall
from pinecone import Pinecone, PodSpec
from dotenv import load_dotenv
load_dotenv()
# Setup Pinecone & OpenAI
pc = Pinecone(
api_key=os.environ.get("PINECONE_API_KEY")
)
index_name = 'function-db'
openai.api_key = os.getenv('OPENAI_API_KEY')
EMBEDDIG_MODEL = "text-embedding-ada-002"
if 'function-db' not in pc.list_indexes().names():
print('Creating index')
pc.create_index(index_name, dimension=1536, metric='cosine', spec=PodSpec(
environment='eu-west1-gcp',
pod_type='p1.x1',
pods=1,
shards=1,
))
index = pc.Index(index_name)
def insert_functions(function_list: [dict]):
print(function_list)
function_descriptions = [func['function']['description'] for func in function_list]
# Embed the function descriptions
embeddings = openai.embeddings.create(
input=function_descriptions,
model=EMBEDDIG_MODEL
).data
docs = []
for i, func in enumerate(function_list):
doc = (
func['function']['name'], # We use the function name as the ID
embeddings[i].embedding, # Embedding
{'name': func['function']['name'], # Additional metadata, we'll use the function later
'description': function_descriptions[i],
'function': json.dumps(func)
}
)
docs.append(doc)
print(f'Inserting {len(docs)} functions into the index.')
index.upsert(docs)
# Loading function descriptions from JSON
file_path = 'functions.json' # <-- If you store your function descriptions in a JSON file
with open(file_path, 'r') as file:
data = json.load(file)
insert_functions(data)4. Retrieve functions
Now that we got the functions stored in the Vector DB we can retrieve them like this, using the user query as the search query.
def get_functions(query: str, k: int=2, embedding_model=EMBEDDIG_MODEL):
embedding = openai.embeddings.create(
input=query,
model=embedding_model
).data[0].embedding
result = index.query(
vector=embedding,
top_k=k,
include_metadata=True
)
return result
search_result = get_functions("Add 2.3495 and 24.9593", 2)
functions_to_use = [json.loads(result.metadata['function']) for result in search_result.matches]
print(functions_to_use)We have now successfully retrieved the most semantically similar functions to the users query. One thing to point out here is that this search method can be done in a lot of different ways. You don’t need to use the user query to search for the functions, and you can leverage similar techniques used within RAG systems, such as HyDe (Hypothetical Document Embeddings) and Multi-Query.
5. Pass the functions to ChatGPT
response = openai.chat.completions.create(
model='gpt-4-1106-preview',
messages=history,
tools=functions_to_use # <-- Add the functions as the list of tools
)Optional: If you want, you can also include a “search_more_tools” function, which is injected after retrieval. This gives ChatGPT the option to search for more functions if it does not find any of the functions relevant in the first attempt. I think of this like clicking the next page in google search.
6. Run the functions
Now we need to make it all come together so we are able to run the functions ChatGPT decided to run. We also need to create some more logic to make the user able to talk to ChatGPT and invoke all the functions after one another until ChatGPT has found its final response. I have included all the code for this setup below with comments.
First we create run_function which can run a tool call from ChatGPT. Then we setup the conversation flow with ChatGPT including looping through tool calls, before the model outputs its final answer. All this is run in a while loop where the questions are added to list of messages at the bottom.
def run_function(function_call: ChatCompletionMessageToolCall):
print('Running function:', function_call.function.name, 'with args:', function_call.function.arguments)
try:
function_name = function_call.function.name
function_to_call = function_map[function_name]
function_args = json.loads(function_call.function.arguments)
return function_to_call(function_args)
except Exception as e:
print('Error:', e)
# Handle your exeptions such as invalid function_name etc.
def run_query(query: str, history: [dict]):
# Step 1: get the functions to use
functions = get_functions(query=query, k=1).matches
functions_to_use = [json.loads(func['metadata']['function']) for func in functions] #+ [get_more_functions]
while True: # Loop until the model decides to stop
# Step 2: call the model
response = openai.chat.completions.create(
model='gpt-4-1106-preview',
messages=history,
tools=functions_to_use,
)
choice = response.choices[0]
tool_calls = choice.message.tool_calls
# Step 3: check if the model wanted to call tools (functions)
if tool_calls:
messages.append(choice.message) # extend conversation with assistant's reply
# Step 4: call the tools
for tool_call in tool_calls:
tool_answer = run_function(tool_call)
function_name = tool_call.function.name
messages.append(
{
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": str(tool_answer),
}
)
else:
print('Finished evaluation')
messages.append({
'role': choice.message.role,
'content': choice.message.content
})
return choice.message.content
# Running the app
# Include a system prompt that highlights the ability of function calling
messages = [{'role': 'system',
'content': 'You are an AI with access to various tools to solve problems. '
'You can use these tools and decide in which order to use them. '
'You may face complex problems requiring you to use tools either one after the other or at the same time. '
'You should not assume the outcome of using a tool without actually using it. '
'If needed, you can access more tools to help solve the problem.'}]
while True:
query = input('Ask question: ')
messages.append({'role': 'user', 'content': query})
answer = run_query(query, messages)
print(answer)And thats it!
Running this will start a conversation with ChatGPT where we retrieve functions from our Function DB before adding them with the query we send to ChatGPT. ChatGPT will then invoke the functions it sees fit, and run functions until it is satisfied, or until it realise it’s unable to answer the question. This technique is more powerful when we add more and more functions, and especially when we start to use agents as functions.
