About LiteLLM
LiteLLM is an open-source AI gateway and LLM proxy built for platform teams managing AI workloads. It sits in front of your AI stack to show who’s using which models, cap spend before it runs, and route each request to the right model, with self-hosted deployment in minutes and no credit card required.
Key Features
Platform teams need a single control plane to manage multiple LLMs and prevent runaway costs. LiteLLM provides that with routing, governance, and observability across models and MCPs:
Routing & governance
Route requests to the appropriate model; enforce spend caps before they run; manage access across teams.
Self-hosted deployment
Deploy in minutes on your own infrastructure; data stays in-house; no credit card required.
Model and MCP integrations
Supports OpenAI, Anthropic, Azure, Bedrock, Vertex AI, Mistral, and more; MCPs like Jira, Atlassian, GitHub, GitLab for governance.
Observability & optimization
Monitor usage, latency, and errors; adjust routing to reduce costs and improve response times.
Summary
Best for platform engineering, security/compliance-focused ops, and teams shipping AI at scale.
