Sargo

What it is

A caller POSTs one provider-neutral inference request; sargo forwards it to Anthropic or OpenAI and returns the reply in the same neutral shape. It lives in rust/sargo in the gligabao repository and is not used by gligabao itself yet.

Build and run

./run_build_rust.sh builds the release binary (rust/sargo/target/release/sargo); with --static it builds a fully static musl binary that runs on any Linux of the same CPU architecture (on Ubuntu it needs musl-tools). ./run_server_sargo.sh starts the compiled server and passes extra arguments to it.

# build (needs cargo)
$ ./run_build_rust.sh            # or --static for a musl binary

# run (at least one provider key must be set)
$ ANTHROPIC_API_KEY=... OPENAI_API_KEY=... ./run_server_sargo.sh path/to/config.toml

# or from the project directory
$ cd rust/sargo && cargo run      # reads ./config.toml (optional)

Config

A config.toml read once at start-up: from the first command-line argument, or SARGO_CONFIG, or ./config.toml. Every key is optional; if no path is given and ./config.toml does not exist, the defaults below are used. Unknown keys are rejected.

[server]
address = "127.0.0.1:8431"
read_timeout_secs = 30
max_body_bytes = 10485760

[http_client]
timeout_secs = 300

[providers.anthropic]
base_url = "https://api.anthropic.com"
api_key_env = "ANTHROPIC_API_KEY"

[providers.openai]
base_url = "https://api.openai.com"
api_key_env = "OPENAI_API_KEY"

[server] sets the address, how long to wait for a client that stops sending and the largest request body; [http_client] how long to wait for a model provider (models can be slow); [providers.*] the base URL and the name of the environment variable that holds the API key.

The API

GET /health answers {"status":"ok"}. POST /v1/infer takes a request like this one:

{
  "provider": "anthropic",
  "model": "claude-sonnet-4-5",
  "system": "You are helpful.",
  "messages": [{"role": "user", "content": [{"type": "text", "text": "Weather in Oslo?"}]}],
  "tools": [{"name": "weather", "description": "Get weather",
            "input_schema": {"type": "object", "properties": {"city": {"type": "string"}},
                            "required": ["city"]}}],
  "max_tokens": 1024
}

The reply has the answer as content blocks, the stop reason and the token usage. stop_reason is end_turn, tool_use, max_tokens or the provider's raw value.

{
  "provider": "anthropic", "model": "claude-sonnet-4-5",
  "content": [{"type": "tool_use", "id": "tu_1", "name": "weather", "input": {"city": "Oslo"}}],
  "stop_reason": "tool_use",
  "usage": {"input_tokens": 50, "output_tokens": 20}
}

Errors are {"error": "..."} with 400 (bad request), 502 (the provider failed; may be retried) or 503 (the provider is not configured in sargo). system, tools, max_tokens (default 1024) and temperature are optional.

Tool calling

sargo is stateless: the caller runs the loop. Repeat until stop_reason is end_turn.

JSON Schema

rust/sargo/inference.schema.json (JSON Schema draft 2020-12) describes the whole API for client authors. The root is the POST /v1/infer request; $defs holds the reply, error and health bodies, and x-endpoints lists each endpoint with its request and reply schemas. Validate requests against it before sending. A unit test keeps its property names in step with the Rust structs.

Go client

The package internal/sargo is a client for the server: New, Client.Infer and Client.Health. Its types follow the JSON Schema, and a request is checked against the schema's rules before it is sent, so a bad one fails in Go instead of with a 400. Server errors come back as *sargo.APIError (Retryable() is true for 502); Response.AssistantMessage() helps with the tool loop.

c, err := sargo.New("http://127.0.0.1:8431")
resp, err := c.Infer(ctx, &sargo.Request{
	Provider: sargo.ProviderAnthropic,
	Model:    "claude-sonnet-4-5",
	Messages: []sargo.Message{{Role: sargo.RoleUser, Content: []sargo.ContentBlock{sargo.TextBlock("Hello")}}},
})
fmt.Println(resp.Text())

Swapping parts