Providers

Tapioca speaks to Ollama, Anthropic, Bedrock and Vertex AI (Anthropic models on AWS/GCP), Azure OpenAI, Gemini (via Google's OpenAI-compatible endpoint), and any other OpenAI-compatible server — LM Studio, vLLM, llama.cpp, OpenRouter, OpenAI itself. All stream, all support tool calls and thinking where the backend does.

Providers are entries under [providers.*] in the config; the entry name is what you use in /model provider:name and --model. default_provider picks which one new agents use.

Ollama

[providers.ollama]
type = "ollama"
base_url = "http://localhost:11434"

This is the default — with Ollama running locally, Tapioca works out of the box. Leave default_model empty to use the first model the server reports.

Anthropic

[providers.anthropic]
type = "anthropic"
api_key_env = "ANTHROPIC_API_KEY"

api_key_env names an environment variable holding the key. An inline api_key = "sk-ant-..." also works but is discouraged — the config file is plain TOML on disk.

AWS Bedrock

[providers.bedrock]
type = "bedrock"
region = "us-east-1"
profile = "default"

Anthropic models on AWS. Credentials come from the usual AWS environment variables or ~/.aws/credentials (profile selects a shared-credentials profile); the model name is a Bedrock id.

Google Vertex AI

[providers.vertex]
type = "vertex"
project = "my-gcp-project"
region = "us-east5"
credentials_file = "/path/to/service-account.json"

Anthropic models on GCP. Auth is a service account key, a GOOGLE_ACCESS_TOKEN, or whatever gcloud is logged in as.

Gemini

[providers.gemini]
type = "gemini"
api_key_env = "GEMINI_API_KEY"

Google Gemini via its OpenAI-compatible endpoint.

Azure OpenAI

[providers.azure]
type = "azure"
base_url = "https://<resource>.openai.azure.com"
api_key_env = "AZURE_OPENAI_API_KEY"
api_version = "2024-10-21"

base_url is your resource, and the model name is a deployment. api_version defaults to a known-good one.

OpenAI-compatible servers

[providers.lmstudio]
type = "openai"
base_url = "http://localhost:1234"
context_window = 32768

type = "openai" covers OpenAI, LM Studio, vLLM, OpenRouter and anything else that speaks the OpenAI chat API. context_window feeds the context gauge when the server does not report one.

For a server behind an unusual credential style, type = "custom" carries its own: auth_style of bearer, header, query or none, plus [providers.*.headers] sent on every request.

llama.cpp

llama-server has its own type rather than going through openai, because it serves things the generic type cannot ask for:

[providers.llamacpp]
type = "llamacpp"
base_url = "http://localhost:8080"
# api_key_env = "LLAMA_API_KEY"   # only if started with --api-key

• The context gauge is real. /props reports the context the server actually allocated per slot, which is what generation runs against — a model's trained window says nothing about what this server was started with.
• Tool calls need the chat template. Older builds refuse them unless started with --jinja; recent ones have it on by default. When that is the cause, the error says so rather than reporting a bare 500.
• Models are named by their GGUF basename, not their full path.
• Router mode works. A llama-server fronting several models and loading them on demand has allocated no context until one is loaded — that is not a broken server, and it is no longer reported as one.

It has to be configured, not discovered. Port 8080 is the most contested there is — a Spring Boot app, a Tomcat, a Jenkins or any dev server takes it first — so Tapioca never goes looking for a model server on it. Add the entry above, or add it from /connect.

Connecting and disconnecting

/connect shows every provider Tapioca can reach, with a status that comes from a real call rather than from whether a key is set, so a revoked key reads as broken rather than configured. Selecting an unconfigured one opens a form that tests the credential before writing it.

ctrl+d disconnects the selected provider, removing it from your config. It arms on the first press and acts on the second, because a key entered through that form is stored in the config file itself when no environment variable holds it — for those providers the config is the only copy, and re-adding the entry does not bring the key back.

Costs

The tokens panel estimates session cost from a price table, $ per million tokens, matched by model prefix. Sensible defaults exist for common Anthropic/OpenAI models (fetched from models.dev at startup unless model_catalog = false); override with:

[costs."claude-sonnet"]
in = 3.0
out = 15.0

Switching models

/model opens the picker; /model [provider:]name switches directly; --model sets it for one run. title_model in the config points session-title generation at a cheaper model.