Guides

Tool calling

Let a model call functions you define, then send the results back.

Tool calling works on models that list tools in supported_parameters. Check GET /v1/models or the model's page before relying on it.

Define a tool

curl https://deference.si/v1/chat/completions \
  -H "Authorization: Bearer $DEFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5.5",
    "messages": [{ "role": "user", "content": "What is the weather in Lisbon?" }],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }]
  }'

import json, os
from openai import OpenAI

client = OpenAI(base_url="https://deference.si/v1", api_key=os.environ["DEFERENCE_API_KEY"])

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What is the weather in Lisbon?"}]
response = client.chat.completions.create(model="anthropic/claude-sonnet-5.5", messages=messages, tools=tools)
call = response.choices[0].message.tool_calls[0]
print(call.function.name, json.loads(call.function.arguments))

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://deference.si/v1",
  apiKey: process.env.DEFERENCE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5.5",
  messages: [{ role: "user", content: "What is the weather in Lisbon?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get the current weather for a city.",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
});
console.log(response.choices[0].message.tool_calls);

Read the call

When the model wants a tool, finish_reason is tool_calls and message.tool_calls holds the name and a JSON string of arguments.

Return the result

Append the assistant message, then one tool message per call with its tool_call_id, and send the conversation again.

{
  "role": "tool",
  "tool_call_id": "call_abc123",
  "content": "{\"temperature_c\": 21, \"sky\": \"clear\"}"
}

Control the choice

tool_choice accepts "auto" (default), "none", "required" or a specific function.

On Messages and Responses

The Messages endpoint takes Anthropic's tools and tool_use blocks. The Responses endpoint takes function tools in tools and returns function_call items. Both pass through unchanged.