Kaeso

Most teams wiring LLMs into production end up in one of two places. Some use a hosted gateway like OpenRouter or Portkey, which means your traffic runs through someone else's infrastructure. Others build a homegrown proxy that slowly collects scattered API keys, cost spreadsheets and Slack alerts. Kaeso pitches a third option: "the open-source AI gateway you run yourself." It's a Rust gateway that sits in the request path, a console for keys, routing, limits and logs, and one API your apps call instead of talking to each provider directly. The site promises "nothing new for your apps to learn," which is the right goal for a gateway.

The sharpest contrast with the hosted options is identity. Kaeso lets you connect an OpenID Connect identity provider to each organization, verify your email domains, and require single sign-on or two-factor authentication. All of that is in the MIT-licensed code, and the site says there's "no enterprise edition to buy." Plenty of infrastructure tools put SSO behind a sales call, so this is a real point in Kaeso's favor. Compared with LiteLLM, the other well-known self-hosted proxy, Kaeso leans harder on the console. The demo shows requests and error rates, tokens split into input, cache read, cache write, output and reasoning, and cost broken down by project, model and member. There's also a Playground where you can send a prompt and then find it in the request logs.

The catch is maturity. The latest release is 0.4.2, which lets organization admins add their own providers and rotate keys in one step. Version 0.4.0 added the option to plug in ChatGPT accounts as OpenAI provider keys and Grok accounts as an experimental provider key, so projects can draw on those subscription plans instead of API accounts. That's a clever idea, but it also shows how fast the product is still changing. You'll be operating pre-1.0 software on your own infrastructure, and the public pages don't list which providers or routing rules are supported. If you want full control of your keys and logs and you have someone who can run it, Kaeso is worth a serious look.