Runtime Configuration API¶
These configuration types are shared by both the CLI and the Python library.
Describe how a model reference should be resolved and executed.
This is the shared execution contract used by the CLI, the library, and the local server. Field annotations remain the source of truth for supported options, while the helper methods normalize and validate those fields for planning and execution.
resolved_backend() -> str | None
Return the normalized backend override when provided.
requested_kv_cache_strategy() -> str | None
Return the normalized explicit KV strategy override when one exists.
resolved_strategy_selector_profile() -> str
Return the normalized selector profile.
resolved_kv_cache_strategy() -> str
Return the normalized KV cache strategy.
resolved_kv_cache_lifecycle() -> str
Return the normalized cache lifecycle.
resolved_kv_cache_adaptation_mode() -> str
Return the normalized cache adaptation mode.
resolved_kv_cache_window_tokens() -> int | None
Return the normalized sliding-window token budget.
resolved_dense_projection_chunk_rows() -> int | None
Return the normalized explicit dense-projection chunk row budget.
resolved_offload_cpu_policy() -> str
Return the normalized CPU offload policy.
resolved_adapter_dir() -> Path | None
Return the absolute adapter directory when one is configured.
validate() -> None
Validate the configuration before planning or execution.
Raises:
| Type | Description |
|---|---|
ValueError
|
Raised when any runtime option is structurally invalid, contradictory, or unsupported for the current execution model. |
Describe generation-time sampling and streaming behavior.
Field annotations remain the source of truth for supported sampling controls.
Use :meth:validate before execution when constructing this type directly.
validate() -> None
Validate sampling and generation limits.
Raises:
| Type | Description |
|---|---|
ValueError
|
Raised when token or sampling limits fall outside the supported runtime range. |
sampling_enabled() -> bool
Return whether stochastic sampling is enabled.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
|