ADR-004: Thin Provider Abstraction & Routing
- Status: Accepted
- Date: 2026-08-12
Context & Problem Statement
Integrating commercial and local LLMs often leads to adopting heavy agent frameworks (LangChain, LiteLLM, CrewAI) that pull hundreds of transitive dependencies, obscure HTTP errors, and introduce non-deterministic loops.
Decision
Define a single structural Provider protocol in Python:
- Implement lightweight (~40 line) REST adapters for Anthropic, Gemini, OpenAI, and local OpenAI-compatible endpoints (Ollama/vLLM), alongside a deterministic
FixtureProviderfor offline testing. - Route models via declarative rules in
ai.config.yaml. - Enforce that tasks marked
privacy: privatenever fall back to cloud providers.