Struct OpenAiCompatibleModelProvider 

pub struct OpenAiCompatibleModelProvider {
    pub alias: String,
    pub name: String,
    pub base_url: String,
    pub credential: Option<String>,
    pub auth_header: AuthStyle,
    /* private fields */
}

A model_provider that speaks the OpenAI-compatible chat completions API. Used by: Venice, Vercel AI Gateway, Cloudflare AI Gateway, Moonshot, Synthetic, OpenCode Zen, OpenCode Go, Z.AI, GLM, MiniMax, Bedrock, Qianfan, Groq, Mistral, xAI, etc.

Fields

alias: String

[providers.models.<alias>] key this provider was constructed under. Used by the Attributable impl so log emissions carry the real composite (<type>.<alias>) instead of the bare type.

name: String

base_url: String

credential: Option<String>

auth_header: AuthStyle

Implementations

impl OpenAiCompatibleModelProvider

pub fn new( alias: &str, name: &str, base_url: &str, credential: Option<&str>, auth_style: AuthStyle, ) -> Self

pub fn new_with_vision( alias: &str, name: &str, base_url: &str, credential: Option<&str>, auth_style: AuthStyle, supports_vision: bool, ) -> Self

pub fn new_with_user_agent( alias: &str, name: &str, base_url: &str, credential: Option<&str>, auth_style: AuthStyle, user_agent: &str, ) -> Self

Create a model_provider with a custom User-Agent header.

Some model_providers (for example Kimi Code) require a specific User-Agent for request routing and policy enforcement.

pub fn new_with_user_agent_and_vision( alias: &str, name: &str, base_url: &str, credential: Option<&str>, auth_style: AuthStyle, user_agent: &str, supports_vision: bool, ) -> Self

pub fn new_merge_system_into_user( alias: &str, name: &str, base_url: &str, credential: Option<&str>, auth_style: AuthStyle, ) -> Self

For model_providers that do not support role: system (e.g. MiniMax). System prompt content is prepended to the first user message instead.

pub fn with_local_model_tool_sanitize(self) -> Self

Opt this provider into per-model conservative tool-schema sanitization. Today the only trigger is the gemma-4 family on llama.cpp, where the upstream tool-call parser rejects empty-properties / non-string default values. The check runs at convert-time against the runtime model id (not against the family) so the same provider instance happily serves llama, qwen, etc. without sanitization.

pub fn with_unauthenticated_model_listing(self) -> Self

pub fn without_native_tools(self) -> Self

Disable native tool calling, forcing prompt-guided tool use instead.

pub fn with_merge_system_into_user(self) -> Self

Merge all system messages into the first user message before sending. Unlike new_merge_system_into_user, this preserves native tool calling.

pub fn with_timeout_secs(self, timeout_secs: u64) -> Self

Override the HTTP request timeout for LLM API calls.

pub fn with_extra_headers(self, headers: HashMap<String, String>) -> Self

Set extra HTTP headers to include in all API requests.

pub fn with_reasoning_effort(self, reasoning_effort: Option<String>) -> Self

Set reasoning effort for GPT-5/Codex-compatible chat-completions APIs.

pub fn with_api_path(self, api_path: Option<String>) -> Self

Set a custom API path suffix for this model_provider. When set, replaces the default /chat/completions path.

pub fn with_max_tokens(self, max_tokens: Option<u32>) -> Self

Set the maximum output tokens for API requests.

pub fn with_models_dev_key(self, key: &str) -> Self

Set the models.dev catalog key for this model_provider. When set, list_models returns the catalog’s model list for that key.

pub fn with_openrouter_vendor_prefix(self, prefix: &str) -> Self

Set the OpenRouter vendor prefix for this model_provider (e.g. "x-ai", "tencent", "rekaai"). list_models falls back to this catalog when neither a credential nor a working models.dev entry is available.

Trait Implementations

impl Attributable for OpenAiCompatibleModelProvider

fn role(&self) -> Role

fn alias(&self) -> &str

impl Clone for OpenAiCompatibleModelProvider

fn clone(&self) -> OpenAiCompatibleModelProvider

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ModelProvider for OpenAiCompatibleModelProvider

fn capabilities(&self) -> ProviderCapabilities

Query model_provider capabilities.

fn list_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<String>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Fetch the list of available model IDs for this model_provider.

fn chat_with_system<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, system_prompt: Option<&'life1 str>, message: &'life2 str, model: &'life3 str, temperature: Option<f64>, ) -> Pin<Box<dyn Future<Output = Result<String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

One-shot chat with optional system prompt. See simple_chat for the temperature contract.

fn chat_with_history<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, messages: &'life1 [ChatMessage], model: &'life2 str, temperature: Option<f64>, ) -> Pin<Box<dyn Future<Output = Result<String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Multi-turn conversation. See simple_chat for the temperature contract.

fn chat_with_tools<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, messages: &'life1 [ChatMessage], tools: &'life2 [Value], model: &'life3 str, temperature: Option<f64>, ) -> Pin<Box<dyn Future<Output = Result<ProviderChatResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Chat with tool definitions for native function calling support. See simple_chat for the temperature contract.

fn chat<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, request: ProviderChatRequest<'life1>, model: &'life2 str, temperature: Option<f64>, ) -> Pin<Box<dyn Future<Output = Result<ProviderChatResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Structured chat API for agent loop callers. See simple_chat for the temperature contract.

fn supports_native_tools(&self) -> bool

Whether model_provider supports native tool calls over API.

fn supports_streaming(&self) -> bool

Whether model_provider supports streaming responses.

fn supports_streaming_tool_events(&self) -> bool

Whether model_provider can emit structured tool-call stream events.

fn stream_chat( &self, request: ProviderChatRequest<'_>, model: &str, temperature: Option<f64>, options: StreamOptions, ) -> BoxStream<'static, StreamResult<StreamEvent>>

Structured streaming chat interface. See simple_chat for the temperature contract.

fn stream_chat_with_system( &self, system_prompt: Option<&str>, message: &str, model: &str, temperature: Option<f64>, options: StreamOptions, ) -> BoxStream<'static, StreamResult<StreamChunk>>

Streaming chat with optional system prompt. See simple_chat for the temperature contract.

fn stream_chat_with_history( &self, messages: &[ChatMessage], model: &str, temperature: Option<f64>, options: StreamOptions, ) -> BoxStream<'static, StreamResult<StreamChunk>>

Streaming chat with history. See simple_chat for the temperature contract.

fn warmup<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Warm up the HTTP connection pool.

fn default_temperature(&self) -> f64

Temperature used when the caller passes None. Override per family.

fn default_max_tokens(&self) -> u32

Max output tokens used when the caller / config doesn’t set one.

fn default_timeout_secs(&self) -> u64

HTTP timeout (seconds) used when the caller / config doesn’t set one.

fn default_base_url(&self) -> Option<&str>

Canonical public API endpoint, when there is one. Returned as a string slice so model_provider impls can serve from const &'static strs without allocations. None = model_provider has no universal endpoint (local model_providers, auth-less CLIs, user-BYO endpoints).

fn default_wire_api(&self) -> &str

Wire protocol variant. Either "responses" (OpenAI Codex-style) or "chat_completions" (everything else). Providers override to their native format.

fn convert_tools(&self, tools: &[ToolSpec]) -> ToolsPayload

Convert tool specifications to provider-native format.

fn simple_chat<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, message: &'life1 str, model: &'life2 str, temperature: Option<f64>, ) -> Pin<Box<dyn Future<Output = Result<String, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Simple one-shot chat (single user message, no explicit system prompt).

fn supports_vision(&self) -> bool

Whether model_provider supports multimodal vision input.