LLM Configuration

Under the hood, cleankoda relies on LiteLLM as an abstraction layer. This gives you maximum flexibility when choosing your AI models: you can use virtually any provider and model supported by LiteLLM (e.g., Anthropic, OpenAI, Mistral, Google Gemini, Groq, Bedrock, or local instances via Ollama).

In addition, any OpenAI-compatible provider (such as llama.cpp, vLLM, LM Studio, or custom API endpoints) can be seamlessly integrated.

Configuration is handled interactively directly within the cleankoda TUI using slash commands:

1. Set Provider (/provider)

Enter the /provider command in the cleankoda command line to select your desired LLM provider:

/provider

Upon selection, you will be prompted to enter the corresponding API key for this provider.

2. Select model (/model)

Once the provider is active, select the desired language model using /model:

/model

Configuration storage location (config.json)

The currently selected provider, the active model, and basic settings are persisted in your user directory:

~/.config/cleankoda/config.json

Example structure:

{
  "provider": "mistral",
  "model": "mistral-small-latest",
  "temperature": 0.2,
  "max_tokens": 4096,
}

Custom OpenAI-compatible providers (custom.json)

If you wish to use a custom endpoint or a service that offers an OpenAI-compatible interface (e.g., local serving via vLLM/Ollama or an internal corporate gateway), configure it in the following file:

~/.config/cleankoda/custom.json

{
  "<custom-provider-name>": {
    "name": "Custom LLM",
    "api_base": "https://.../v1",
    "models": [
      "gemma4-12b",
      "qwen-3.5"
    ],
    "requires_api_key": true
  },

Once these entries are present in custom.json, your custom provider and its models will automatically appear in the selection dialogs for /provider and /model.