Tools/Verified 14 Nov 2026

An open-source inference engine and server used to serve open models at higher throughput, typically on your own GPUs or in a cluster.

Pricing

Free and open source. You pay for GPUs and hosting. Latest non-rc GitHub release seen: v0.32.0 on 18 Oct 2026. A v0.33.0rc3 tag was published 2 Nov 2026.

Platforms

CLI, Local, Cloud

License

Open source

Category

Local models