yera.cli.setup_handlers.llama_cpp

Llama-cpp provider setup handler.

Symbols

class LlamaCppSetup — Handler for configuring Llama-cpp local model providers.

LlamaCppSetup

Inherits: BaseProviderSetup

Handler for configuring Llama-cpp local model providers.

This class manages the discovery of local directories containing GGUF model files, allowing Yera to locate and load local LLMs from common installation paths or user-defined locations.

Methods

detect_config — Attempts to automatically detect local model directories.
ask_for_config — Interactively prompts the user to define local model directories.
validate — Validates that the configured model directories exist on the filesystem.

LlamaCppSetup.detect_config

detect_config() → LlamaCppConnection | None

Attempts to automatically detect local model directories.

Checks for the existence of standard model storage paths (HuggingFace, LM Studio) and user-defined environment variables.

Returns

type: LlamaCppConnection | None

A LlamaCppConnection instance containing detected paths if any are found; otherwise, None.

LlamaCppSetup.ask_for_config

ask_for_config() → LlamaCppConnection | None

Interactively prompts the user to define local model directories.

Displays currently detected paths and enters a loop allowing the user to manually add additional directories. Paths are expanded (e.g., ~) and resolved to absolute paths before being saved.

Returns

type: LlamaCppConnection | None

A LlamaCppConnection instance containing the union of detected and manually added directories, or None if the user declines.

LlamaCppSetup.validate

validate(
    config: LlamaCppConnection,
) → None

Validates that the configured model directories exist on the filesystem.

Unlike cloud providers, this method performs a 'soft' validation. If directories are missing or inaccessible, it prints a warning to the console rather than raising an exception, as directories may be mounted later or are optional.

Parameters

config
type: LlamaCppConnection

The LlamaCpp connection configuration to validate.