Groq

Groq runs open-weight models on custom LPU hardware instead of GPUs, and the difference is latency: responses come back several times faster than the same model served conventionally. That matters most where an LLM sits in the middle of a pipeline rather than at the end. A Groq-backed LLM Worker planning prompts for an image model, routing a Conditional Switch, or extracting JSON adds almost nothing to total pipeline time. Llama 3.1 8B Instant is the cheapest model on the marketplace at $0.05/$0.08 per million tokens; Llama 3.3 70B and GPT-OSS 120B are there when you need more. Charged on the tokens Groq reports.