Claude Code with Local Models and OpenRouter: The Complete Guide

Claude Code is the most agentic coding CLI Anthropic ships — but it does not have to talk to Anthropic. Because Claude Code speaks the standard Anthropic Messages API, you can point it at any compatible endpoint with a single environment variable. That means OpenRouter, your local Ollama server, a private LiteLLM proxy, or a self-hosted model behind your own router all work without forking the source.

This guide walks through the three setup paths, which models actually deliver usable tool-use at each tier, what breaks, and when you should stop and switch back to the official API.

All demos below run from ~/projects/demo/ — a throwaway Node.js project I keep around for exactly this kind of thing. Clone or cp -r it anywhere, the path does not matter.

What Claude Code actually is

Claude Code is a Node-based CLI that runs an agentic loop against the Anthropic Messages API. On every turn it sends a structured request — system prompt, conversation history, and a list of available tools — and the model returns either text or a tool call. Claude Code then executes the tool locally (read a file, run a command, edit a buffer) and feeds the result back into the next turn. The loop continues until the model emits a final text response with no further tool calls.

The protocol is documented and stable. That single fact is the unlock for everything in this post: the same wire format that talks to api.anthropic.com can talk to anything that mimics it. Three concrete implications:

1. Any provider exposing an Anthropic-compatible API works — OpenRouter does this transparently, plus most community proxies.
2. Tool-use is the hard part, not chat. Local models that ace MMLU often still fail Claude Code’s specific tool-call format.
3. System prompt and tool schema come from Claude Code itself. You are not re-prompting the model with your own agent scaffold — the model is being asked to behave as Claude does, on a different backend.

Why swap the backend at all

Five reasons people run Claude Code on non-Anthropic backends, in order of how often I see them:

1. Cost ceiling. A long agentic session can rack up ten or twenty dollars in Claude API spend. OpenRouter lets you cap the model at a budget tier — deepseek/deepseek-chat-v3.1 is roughly a tenth the cost of Sonnet for many tasks.
2. Privacy. Some codebases must never leave the building. Running Claude Code against an on-prem Ollama server keeps every byte on the LAN.
3. Model experiments. You want to A/B Claude against GPT-5, Gemini 2.5 Pro, or your own fine-tune without leaving the same workflow.
4. Rate-limit fall-back. When the Anthropic API throttles you, an OpenRouter request to the same model class often sails through.
5. Capability tail. Different models win on different tasks. Claude is the all-rounder; Qwen3-Coder is arguably sharper on long multi-file refactors; Gemini has a million-token context window.

The three setup paths

Path 1: environment variables (quickest)

Claude Code reads three env vars on every invocation: ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, and ANTHROPIC_MODEL. Override them in your shell and the next claude call goes wherever you pointed it.

# ~/projects/demo — point Claude Code at OpenRouter for one session
export ANTHROPIC_BASE_URL="https://openrouter.ai/api/v1"
export ANTHROPIC_AUTH_TOKEN="${OPENROUTER_API_KEY}"
export ANTHROPIC_MODEL="anthropic/claude-sonnet-4.5"

claude --model "anthropic/claude-sonnet-4.5" "summarize README.md"

That is the whole integration for the trivial case. No code changes, no extra process. The downside: the settings vanish the moment your shell closes. For anything durable, use the next path.

Path 2: ~/.claude/settings.json (durable, project-scoped)

Claude Code looks for a settings file at ~/.claude/settings.json (user-global) and .claude/settings.json (project-scoped). Inside, an env block sets the same three variables persistently.

{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "env": {
    "ANTHROPIC_BASE_URL": "https://openrouter.ai/api/v1",
    "ANTHROPIC_AUTH_TOKEN": "sk-or-v1-xxxxxxxxxxxxxxxxxxxx",
    "ANTHROPIC_MODEL": "anthropic/claude-sonnet-4.5",
    "ANTHROPIC_SMALL_FAST_MODEL": "anthropic/claude-haiku-4.5"
  },
  "model": "opusplan"
}

The project-scoped file is the one you want. Commit it to the repo (minus the real key) and every developer on the team gets the same backend. Pair it with a .env.local for the actual token and a direnv or dotenv loader.

A few other keys worth knowing about:

• ANTHROPIC_SMALL_FAST_MODEL — model used for the lightweight “haiku-class” tasks Claude Code spawns in the background. Save money by pointing this at a cheaper model.
• DISABLE_PROMPT_CACHING — turn this on when the target provider does not implement Anthropic’s prompt-cache format (OpenRouter has partial support; local servers do not).
• permissions — explicit allow/deny lists for tools Claude Code can use. Always set this when you are routing to a non-Anthropic model: weaker models will occasionally call tools they should not.

Path 3: claude-code-router (multi-model fan-out)

When you want different backends for different sub-tasks — Sonnet for the main loop, a local Qwen for background summarization, Gemini for the long-context planning step — a single env-var override is not enough. The community claude-code-router project (and the similar claude-code-proxy) intercepts Claude Code’s HTTP calls and rewrites them per-task.

{
  "PORT": 3456,
  "Providers": [
    {
      "name": "openrouter",
      "api_base_url": "https://openrouter.ai/api/v1/chat/completions",
      "api_key": "sk-or-v1-xxxxxxxxxxxxxxxxxxxx",
      "models": [
        "anthropic/claude-sonnet-4.5",
        "anthropic/claude-haiku-4.5",
        "deepseek/deepseek-chat-v3.1"
      ]
    },
    {
      "name": "ollama",
      "api_base_url": "http://192.168.1.42:11434/v1/chat/completions",
      "api_key": "ollama",
      "models": ["qwen3:14b", "gemma3:12b"]
    }
  ],
  "Router": {
    "default":     "openrouter,anthropic/claude-sonnet-4.5",
    "background":  "ollama,qwen3:14b",
    "think":       "openrouter,deepseek/deepseek-chat-v3.1",
    "longContext": "openrouter,google/gemini-2.5-pro"
  }
}

The router runs as a local HTTP server on a chosen port; you then point ANTHROPIC_BASE_URL at http://127.0.0.1:3456. The router decides — per request — which upstream to forward to, based on the request metadata (model name, prompt size, whether the request was tagged “think” or “background”).

This is the path I reach for when a team has more than one use case for the same Claude Code install. The maintenance cost is real, though: the router is a third-party Node project, version-pinned, and you will need to update it independently of Claude Code.

Path A: OpenRouter in practice

OpenRouter is the easiest non-Anthropic target. It exposes an OpenAI-shaped endpoint, an Anthropic-shaped endpoint, and a unified API for ~200 models. For Claude Code specifically you use the Anthropic-shaped endpoint, which means tool-use works exactly as it does on the official API — no schema translation, no edge cases.

// ~/.claude/settings.json — production-grade OpenRouter config
{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "env": {
    "ANTHROPIC_BASE_URL": "https://openrouter.ai/api/v1",
    "ANTHROPIC_AUTH_TOKEN": "sk-or-v1-xxxxxxxxxxxxxxxxxxxx",
    "ANTHROPIC_MODEL": "anthropic/claude-sonnet-4.5",
    "ANTHROPIC_SMALL_FAST_MODEL": "anthropic/claude-haiku-4.5",
    "DISABLE_PROMPT_CACHING": "1"
  },
  "permissions": {
    "allow": ["Read", "Edit", "Bash(npm:*)", "Bash(pytest:*)"],
    "deny":  ["Bash(rm:*)", "Bash(curl:*)"]
  }
}

Model selection on OpenRouter is the part that matters. Not every model handles Claude Code’s tool-use format well, even when the provider claims function-calling support. Here is the working set as of mid-2026:

{
  "data": [
    { "id": "anthropic/claude-sonnet-4.5",        "ctx": 200000, "tools": true,  "notes": "best quality, $3/$15 per MTok" },
    { "id": "anthropic/claude-haiku-4.5",         "ctx": 200000, "tools": true,  "notes": "fast + cheap, $1/$5 per MTok" },
    { "id": "openai/gpt-5",                        "ctx": 128000, "tools": true,  "notes": "tool-use via OpenAI schema" },
    { "id": "google/gemini-2.5-pro",               "ctx": 1000000,"tools": true,  "notes": "huge context, function-calling" },
    { "id": "deepseek/deepseek-chat-v3.1",         "ctx": 128000, "tools": true,  "notes": "open-weights, $0.27/$1.10" },
    { "id": "meta-llama/llama-3.3-70b-instruct",   "ctx": 131072, "tools": true,  "notes": "open-weights via inference" },
    { "id": "qwen/qwen3-coder",                    "ctx": 262144, "tools": true,  "notes": "purpose-built for agents" }
  ]
}

Two things to watch for on OpenRouter:

• Prompt caching is partial. OpenRouter’s Anthropic-compatible endpoint supports cache reads/writes for native Anthropic models, but the response is sometimes flaky. DISABLE_PROMPT_CACHING=1 removes the ambiguity at a small cost-per-token penalty.
• Streaming tool calls work, but the first chunk can be slow on cold paths. If your session feels sluggish, set CLAUDE_CODE_MAX_STREAM_RETRIES=2.

Path B: Ollama on your local box

Running Claude Code against a local Ollama server is the path that gets talked about the most and works the least reliably. The reason is purely the tool-use format: Claude Code emits Anthropic-flavored tool schemas, and a small local model that was never trained on that schema will either ignore the tools, hallucinate tools that do not exist, or output malformed JSON.

It does work — with the right model. From my testing on a Windows box with a Ryzen 7 + 32 GB RAM (CPU-only, no GPU):

The “Mostly works” verdict means: 70-90% of tool calls succeed on a 5-step task. The “Reliable” verdict means: 95%+ on the same task. Anything below “Mostly works” is a frustrating experience.

The single biggest lever is the system prompt. Ollama lets you pin a Modelfile system prompt that the model sees on every request. With Claude Code, you do not control the system prompt directly — it is generated by the CLI. What you can do is set a per-model preamble that primes the model for tool-use:

# Ollama Modelfile for Claude Code tool-use
# Build: ollama create claude-code-local -f Modelfile
FROM qwen3:14b

SYSTEM """You are a coding assistant. When the user asks you to act on files,
use the provided tools exactly as specified. Do not invent tool names.
Always read a file before editing it. Prefer small, focused diffs."""

PARAMETER temperature 0.2
PARAMETER num_ctx 32768
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Build it with ollama create claude-code-local -f Modelfile, then point Claude Code at the new tag.

A practical gotcha: Ollama’s default context window is 2048 tokens. Claude Code regularly sends prompts of 20k+. You must set num_ctx in the Modelfile (or via the OLLAMA_NUM_CTX env var) to 32768 minimum, ideally 65536. Smaller contexts cause silent truncation that looks like the model “forgetting” tools.

Path C: LiteLLM as a custom proxy

If you want OpenRouter’s flexibility without OpenRouter’s egress (or you are behind a network policy that blocks openrouter.ai), LiteLLM is the answer. It is a single Python process that exposes one OpenAI-shaped endpoint and translates it to dozens of upstream providers — Anthropic, OpenAI, Ollama, vLLM, Bedrock, Vertex, anything.

# litellm config -- single OpenAI-compatible endpoint in front of mixed backends
model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4.5
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: qwen-local
    litellm_params:
      model: openai/qwen3:14b
      api_key: ollama
      api_base: http://192.168.1.42:11434/v1

  - model_name: gemma-local
    litellm_params:
      model: openai/gemma3:12b
      api_key: ollama
      api_base: http://192.168.1.42:11434/v1

router_settings:
  num_retries: 2
  timeout: 60
  allowed_fails: 1

general_settings:
  telemetry: False

Run it with litellm --config litellm-config.yaml --port 4000, then point Claude Code at http://127.0.0.1:4000. LiteLLM is heavier than claude-code-router and adds ~100ms of latency, but it is more stable, has retries built in, and survives an upstream provider disappearing mid-session.

A reproducible test matrix

If you want hard numbers for your own setup, run this against whatever models you have available. The script lives in scripts/post-assets/claude-code-with-local-models-and-openrouter/snippets/08-test-matrix.sh in the blog repo, and runs from ~/projects/demo/.

#!/usr/bin/env bash
# test-matrix.sh -- probe Claude Code against a list of backends + models.
# Run from any project dir with `claude` installed.

set -u
PROMPT='List the JS files in src/ and print their line counts.'
LOG="${1:-results.md}"
: > "$LOG"

run() {
  local label="$1"; shift
  {
    echo "## $label"
    echo '```text'
    "$@" 2>&1
    echo '```'
    echo
  } >> "$LOG"
}

# --- OpenRouter (cloud) ---
export ANTHROPIC_BASE_URL="https://openrouter.ai/api/v1"
export ANTHROPIC_AUTH_TOKEN="$OPENROUTER_API_KEY"
run "openrouter / anthropic/claude-sonnet-4.5" \
  claude --model "anthropic/claude-sonnet-4.5" "$PROMPT"

run "openrouter / deepseek/deepseek-chat-v3.1" \
  claude --model "deepseek/deepseek-chat-v3.1" "$PROMPT"

# --- Ollama (local) ---
export ANTHROPIC_BASE_URL="http://192.168.1.42:11434"
export ANTHROPIC_AUTH_TOKEN="ollama"

for m in qwen3:8b qwen3:14b gemma3:4b gemma3:12b; do
  run "ollama / $m" claude --model "$m" "$PROMPT"
done

Pick a prompt that exercises a tool (read, then maybe edit). A “list the JS files in src/ and count lines” task is a good minimum bar — it forces the model to call Bash, parse the output, and return a clean text response. Anything that passes that on three consecutive runs is genuinely usable for Claude Code.

Use cases that justify the swap

• Bootstrapping a private monorepo. Point Claude Code at an on-prem Ollama box, let it read and explore freely, never worry about training-data leakage.
• A/B testing model releases. OpenRouter gives you a new model the day it ships. Swap ANTHROPIC_MODEL, run a fixed eval prompt, compare outputs.
• Cost-capping a junior dev’s session. Route background tasks to a local model and the heavy loop to a budget cloud model. The “background” model in claude-code-router handles the sub-agent summarization calls that burn tokens.
• Forcing tool-use discipline. Local models that hallucinate tools are actually useful for teaching Claude Code’s permission system. When the model tries to call a forbidden tool, you see exactly which ones are misbehaving.
• Long-context code review. Gemini 2.5 Pro on OpenRouter has a 1M-token context window. Point Claude Code at it for repository-wide analysis that would otherwise need summarization.

Limitations and gotchas

Five things that bit me while writing this guide, in order of severity:

1. Tool schema drift. When Anthropic ships a new tool (or a new field on an existing tool), OpenRouter and Ollama do not always update on the same day. A “tool_use_error” on an otherwise-working session is almost always this. Fix: pin a Claude Code version (npm install -g @anthropic-ai/claude-code@<version>).
2. No streaming tool calls on most local servers. Ollama streams chat completions, but the tool-call events are batched at the end. The visible effect: Claude Code “freezes” for a few seconds before the first tool runs. This is a protocol-level issue, not a config issue.
3. Prompt caching. Anthropic’s prompt cache is format-specific. OpenRouter supports it only for native Anthropic models. Local servers do not implement it. Expect higher token costs when you turn caching off.
4. System prompt leakage. Some community models will leak the system prompt verbatim if asked. Not a Claude Code issue, but worth knowing if your internal docs end up in the system prompt.
5. Rate limits. OpenRouter enforces per-model rate limits, often lower than the upstream provider’s own limit. If you hit a 429, switch models rather than waiting.

When to stop and switch back

The decision is straightforward once you internalize the trade-offs:

• Stay on the Anthropic API when the task is large, multi-file, and you trust the codebase. Sonnet 4.5’s tool-use is the highest-quality in the field. The cost is real but predictable.
• Switch to OpenRouter + Claude via OpenRouter when you want cost ceilings or want to A/B Claude against another model. You get the same quality for a few percent extra per token.
• Switch to OpenRouter + a non-Claude model when the task is repetitive (boilerplate generation, test scaffolding) and a budget model is good enough.
• Switch to Ollama local when privacy is non-negotiable or you want zero per-token cost. Accept that you will spend time on tool-use prompt engineering.
• Skip the swap entirely for any task that involves more than ~10 tool calls in a session. The latency and tool-use reliability of non-Anthropic models degrades quickly past that threshold on every backend I tested.

Verdict

The plumbing to swap Claude Code’s backend is one environment variable. The hard part is choosing the right model for the right task. After a week of running it against OpenRouter, Ollama, and LiteLLM, my default config is:

• Main loop: anthropic/claude-sonnet-4.5 via OpenRouter — same quality as the official API, slight markup, no surprises.
• Background tasks: local qwen3:14b via Ollama on a homelab box — good enough for sub-agent summarization, zero per-token cost.
• Long-context code review: google/gemini-2.5-pro via OpenRouter — when I need to paste in a 600k-token repo and ask Claude Code to find every TODO.

That setup covers about 90% of what I do. The remaining 10% I send to the official Anthropic API and accept the cost, because the task warrants it.

Claude Code was never meant to be a single-model tool. Treat it as an agent harness, and pick the brain per task.