> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/nullclaw/nullclaw/llms.txt
> Use this file to discover all available pages before exploring further.

# Providers

> AI model provider integration via vtable interface

## Overview

Providers are NullClaw's abstraction layer for AI model APIs. Every provider implements the `Provider` vtable interface, enabling runtime-swappable model backends with zero code changes.

## Provider Interface

```zig theme={null}
pub const Provider = struct {
    ptr: *anyopaque,
    vtable: *const VTable,

    pub const VTable = struct {
        // Simple one-shot: system prompt + user message → text
        chatWithSystem: *const fn(
            ptr: *anyopaque,
            allocator: std.mem.Allocator,
            system_prompt: ?[]const u8,
            message: []const u8,
            model: []const u8,
            temperature: f64,
        ) anyerror![]const u8,

        // Structured chat: full conversation → ChatResponse
        chat: *const fn(
            ptr: *anyopaque,
            allocator: std.mem.Allocator,
            request: ChatRequest,
            model: []const u8,
            temperature: f64,
        ) anyerror!ChatResponse,

        // Capabilities
        supportsNativeTools: *const fn(ptr: *anyopaque) bool,
        getName: *const fn(ptr: *anyopaque) []const u8,
        deinit: *const fn(ptr: *anyopaque) void,

        // Optional: streaming, vision, native tools
        supports_streaming: ?*const fn(ptr: *anyopaque) bool = null,
        supports_vision: ?*const fn(ptr: *anyopaque) bool = null,
        stream_chat: ?*const fn(...) anyerror!StreamChatResult = null,
        chat_with_tools: ?*const fn(...) anyerror!ChatResponse = null,
    };
};
```

## Supported Providers

### Core Providers (9)

These providers have dedicated implementations in `src/providers/`:

| Provider       | Models                               | Features                                           |
| -------------- | ------------------------------------ | -------------------------------------------------- |
| **Anthropic**  | Claude Opus/Sonnet/Haiku 3/3.5/4     | Native tools, streaming, vision, extended thinking |
| **OpenAI**     | GPT-4o, GPT-4, GPT-3.5, o1/o3        | Native tools, streaming, vision, reasoning effort  |
| **OpenRouter** | 200+ models (proxy)                  | Native tools, streaming, vision (model-dependent)  |
| **Ollama**     | Local models (llama3, mistral, etc.) | Native tools, streaming, local inference           |
| **Gemini**     | Gemini Pro/Flash/Ultra               | Native tools, streaming, vision, thinking          |
| **Venice**     | Privacy-focused models               | No logs, Swiss jurisdiction                        |
| **xAI**        | Grok models                          | Real-time data, Twitter integration                |
| **Groq**       | Fast inference (llama3, mixtral)     | Ultra-low latency                                  |
| **Mistral**    | Mistral Large/Medium/Small           | EU-based, open weights                             |

### Compatible Providers (41+)

These providers work via the **OpenAI-compatible adapter** (`src/providers/compatible.zig`):

<AccordionGroup>
  <Accordion title="All Compatible Providers">
    * **Together AI** — Fast inference for open models
    * **Fireworks** — Optimized model deployment
    * **Perplexity** — Search-augmented generation
    * **Cohere** — Enterprise-grade NLP
    * **DeepSeek** — Code-specialized models
    * **Replicate** — Community model hosting
    * **Hugging Face** — Inference API
    * **Cloudflare Workers AI** — Edge inference
    * **AI21 Labs** — Jurassic models
    * **Anyscale** — Ray-based inference
    * **OctoAI** — Optimized model serving
    * **Novita AI** — Affordable inference
    * **Deep Infra** — GPU cloud
    * **Lepton AI** — Serverless inference
    * **Cerebras** — Ultra-fast inference chips
    * **Monster API** — Low-cost hosting
    * **Nebius** — EU cloud AI
    * **Scaleway** — EU inference
    * **Hyperbolic** — Fast open models
    * **Lambda** — GPU cloud
    * **Modal** — Serverless containers
    * **Runpod** — GPU marketplace
    * **Avian** — Privacy-first AI
    * **Glhf** — Community models
    * **Kindo** — Enterprise AI
    * **SambaNova** — Fast LLM chips
    * **Featherless** — Lightweight inference
    * **GitHub Models** — Azure-backed playground
    * **LM Studio** — Local model server
    * **Llamafile** — Single-file executables
    * **Jan** — Desktop AI runtime
    * **Text Generation Web UI** — Local web interface
    * **vLLM** — High-throughput inference
    * **TGI (Text Generation Inference)** — Hugging Face server
    * **LocalAI** — OpenAI-compatible local server
    * **Tabby** — Code completion server
    * **Continue** — IDE assistant backend
    * **Amazon Bedrock** — AWS managed models
    * **Azure OpenAI** — Microsoft-hosted OpenAI
    * **Google Vertex AI** — GCP model deployment
    * **Custom endpoint** — Any OpenAI-compatible API
  </Accordion>
</AccordionGroup>

## Configuration

### Basic Setup

Providers are configured in `~/.nullclaw/config.json`:

```json theme={null}
{
  "models": {
    "providers": {
      "openrouter": { "api_key": "sk-or-..." },
      "anthropic": { "api_key": "sk-ant-..." },
      "ollama": { "base_url": "http://localhost:11434" }
    }
  },
  "agents": {
    "defaults": {
      "model": { "primary": "openrouter/anthropic/claude-sonnet-4" }
    }
  }
}
```

### Custom Endpoints

Use `custom:` prefix for any OpenAI-compatible API:

```json theme={null}
{
  "models": {
    "providers": {
      "custom:my-endpoint": {
        "api_key": "token",
        "base_url": "https://my-inference.example.com/v1"
      }
    }
  },
  "agents": {
    "defaults": {
      "model": { "primary": "custom:my-endpoint/my-model" }
    }
  }
}
```

### Model Selection

Model string format: `<provider>/<model>`

```json theme={null}
"model": {
  "primary": "openrouter/anthropic/claude-opus-4",
  "fallback": "openai/gpt-4o"
}
```

## Provider Selection Flow

```mermaid theme={null}
flowchart TD
    Start[Agent needs model] --> Parse[Parse model string]
    Parse --> Extract[Extract provider prefix]
    Extract --> Custom{Is custom:?}
    Custom -->|Yes| CustomProvider[CompatibleProvider]
    Custom -->|No| CoreCheck{Core provider?}
    CoreCheck -->|Yes| CoreProvider[Core Provider Instance]
    CoreCheck -->|No| Compatible[CompatibleProvider]
    CustomProvider --> Init[Initialize with API key]
    CoreProvider --> Init
    Compatible --> Init
    Init --> Ready[Provider Ready]
```

## Capabilities

### Native Tool Calling

Providers that return `supportsNativeTools() == true`:

* Anthropic (function calling API)
* OpenAI (function calling API)
* Gemini (function calling API)
* Ollama (via tool schema)
* OpenRouter (proxies native support)

Other providers fall back to **prompt-based tool simulation**.

### Streaming

Providers implement `stream_chat` vtable method:

```zig theme={null}
pub fn streamChat(
    self: Provider,
    allocator: std.mem.Allocator,
    request: ChatRequest,
    model: []const u8,
    temperature: f64,
    callback: StreamCallback,
    callback_ctx: *anyopaque,
) !StreamChatResult
```

Supported by: Anthropic, OpenAI, Ollama, Gemini, most compatible providers.

### Vision (Multimodal Input)

Providers check `supportsVision()` or `supportsVisionForModel(model)`:

```zig theme={null}
const parts = [_]ContentPart{
    makeTextPart("What's in this image?"),
    makeImageUrlPart("https://example.com/cat.jpg"),
};
const msg = ChatMessage{
    .role = .user,
    .content = "",
    .content_parts = &parts,
};
```

Supported by: GPT-4o, Claude Opus/Sonnet 3.5/4, Gemini Pro Vision, OpenRouter (model-dependent).

## Error Handling

### Automatic Retries

The `ReliableProvider` wrapper (`src/providers/reliable.zig`) adds:

* Exponential backoff (50ms → 200ms → 800ms)
* Rate limit detection (429 status)
* Transient error classification
* Circuit breaker integration

### Fallback Chain

Configure fallback models in agent config:

```json theme={null}
"model": {
  "primary": "openrouter/anthropic/claude-opus-4",
  "fallback": "openai/gpt-4o",
  "fallback_on_error": true
}
```

<Warning>
  Fallback triggers only on **provider errors** (rate limits, outages). It does NOT trigger on LLM refusals or content policy violations.
</Warning>

## Performance

### Provider Warmup

Optional vtable method `warmup()` pre-establishes connections:

```zig theme={null}
provider.warmup();
```

Currently a no-op for all providers (HTTP clients are stateless).

### Response Caching

SQLite-backed cache deduplicates identical requests:

```json theme={null}
{
  "memory": {
    "response_cache": {
      "enabled": true,
      "ttl_minutes": 60,
      "max_entries": 10000
    }
  }
}
```

Cache key: `sha256(system_prompt + messages + model + temperature)`

## Implementation Guide

### Minimal Provider

```zig:src/providers/my_provider.zig theme={null}
const std = @import("std");
const Provider = @import("root.zig").Provider;

pub const MyProvider = struct {
    api_key: []const u8,
    base_url: []const u8,

    pub fn chatWithSystem(
        ptr: *anyopaque,
        allocator: std.mem.Allocator,
        system_prompt: ?[]const u8,
        message: []const u8,
        model: []const u8,
        temperature: f64,
    ) anyerror![]const u8 {
        const self: *MyProvider = @ptrCast(@alignCast(ptr));
        // Call API, parse response...
        return try allocator.dupe(u8, "response text");
    }

    pub fn chat(
        ptr: *anyopaque,
        allocator: std.mem.Allocator,
        request: ChatRequest,
        model: []const u8,
        temperature: f64,
    ) anyerror!ChatResponse {
        // Full implementation...
        return .{ .content = "response" };
    }

    pub fn supportsNativeTools(_: *anyopaque) bool {
        return false;
    }

    pub fn getName(_: *anyopaque) []const u8 {
        return "my_provider";
    }

    pub fn deinit(_: *anyopaque) void {}

    pub fn provider(self: *MyProvider) Provider {
        return .{ .ptr = @ptrCast(self), .vtable = &vtable };
    }

    pub const vtable = Provider.VTable{
        .chatWithSystem = chatWithSystem,
        .chat = chat,
        .supportsNativeTools = supportsNativeTools,
        .getName = getName,
        .deinit = deinit,
    };
};
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Configuration" icon="gear" href="/essentials/configuration">
    Full provider configuration reference
  </Card>

  <Card title="Tools" icon="wrench" href="/concepts/tools">
    Learn how tool calling works
  </Card>
</CardGroup>