Modal

If you've ever tried to get a model into production, you know the real work often isn't the model. It's the GPU quota requests, the Kubernetes manifests, the cold starts that make your demo hang for 40 seconds, and the bill for an A100 that sat idle all weekend. Modal is built for exactly that pain. It pitches itself as "cloud infrastructure for teams that develop, train, and serve AI applications at scale," and the core promise is that you write code, and Modal handles capacity, scheduling and scaling underneath it.

The product splits into four pieces that share one runtime. Inference gives you OpenAI-compatible endpoints or lets you "bring your entire inference stack." Training covers everything from single-GPU LoRA runs to multi-node RL, with out-of-the-box support for open source frameworks like slime and Miles. Sandboxes are isolated environments for coding agents and untrusted code, and Modal claims you can create "millions of concurrent sandboxes in under a minute." Functions turn arbitrary code into serverless compute that can fan out over thousands of GPUs or become an HTTPS endpoint that "idles at zero." Under all of it sits a custom container runtime, a CDN-style distributed filesystem for weights and datasets, and a multi-cloud capacity pool.

The honest take: this is one of the more opinionated answers to the "I just want GPUs without the ops" problem, and the per-second billing for GPUs and CPUs means you aren't paying for idle hardware. The tradeoff is that you're buying into Modal's way of packaging and running code, and the pricing material we reviewed didn't include per-GPU rates, so you'll want to run your own workload through it before committing a production budget. If your team spends more time babysitting infrastructure than shipping models, it's worth a serious look.