★★★★★ 4.5 CE

Groq API: Access, Rate Limits & SDKs 2026

GroqCloud is OpenAI-compatible at api.groq.com/openai/v1, so a base-URL swap gives your code a fast, cheap backend for 11 open-weight models at 131k context.

Large Language ModelsVerified today

Full review Pricing Official site ↗

Groq API verdict

Verified today·4 sources checked

GroqCloud's API authenticates with a single bearer GROQ_API_KEY and is OpenAI-compatible at https://api.groq.com/openai/v1.

The official groq SDK, the OpenAI SDK with a base-URL swap, or the Vercel AI SDK, LiteLLM and LangChain all call it with minimal change. It serves 11 open-weight models across Llama, gpt-oss and Qwen at 131k context.

How to build on it

Build on GroqCloud when you want a fast, cheap, OpenAI-shaped backend for open models without changing much code. Keep your OpenAI SDK, swap the base URL to https://api.groq.com/openai/v1 and the key, and pick a model, small gpt-oss for raw speed or Llama 3.3 70B for quality. The same surface covers tool use, structured outputs, prompt caching, speech, reasoning and the agentic Compound system. It ships with production tooling like a checklist, a latency guide, Prometheus metrics and spend limits. Use Flex or Batch tiers to cut cost on deferrable or bulk work, and lean on prompt caching for repeated context. The trade-off is that you are choosing an inference host for open models, so for proprietary frontier capability or context beyond 131k you need a different provider.

Honest limits

GroqCloud is OpenAI-compatible but not identical. Some OpenAI parameters differ, so check the OpenAI Compatibility doc before migrating.
It serves open-weight models only, across Llama, gpt-oss and Qwen, so model quality is the open model's, not a Groq proprietary frontier model.
Throughput is shaped by service tier: on-demand for low latency, Flex for discounted deferred work, Batch for bulk. Pick the tier per workload.
Rate limits are per model and per account. Raise them through tiers, and watch spend limits and per-project model permissions.

Auth: Bearer GROQ_API_KEY
Base URL: api.groq.com/openai/v1
Compat: OpenAI SDK drop-in
SDKs: groq, OpenAI, Vercel, LiteLLM, LangChain
Tiers: On-demand / Flex / Batch

View sources

Build with the Groq API

API code explorer

pip install groq

import os
from groq import Groq

client = Groq(api_key=os.environ.get("GROQ_API_KEY"))
chat_completion = client.chat.completions.create(
    messages=[{"role": "user", "content": "Explain fast inference"}],
    model="llama-3.3-70b-versatile",
)
print(chat_completion.choices[0].message.content)

Response

{
  "choices": [{
    "message": { "role": "assistant", "content": "Fast inference means..." },
    "finish_reason": "stop"
  }],
  "usage": { "prompt_tokens": 8, "completion_tokens": 64 }
}

Cost levers: Flex Processing (discounted deferred work); Batch Processing (bulk async); prompt caching for repeated context.
Handle: 401 (invalid GROQ_API_KEY), 429 (rate limit (per model/account)).

Real Groq code from the official docs. The raw cURL hello-world is the code block above; full limits and endpoints are in the tables on this page.

Your first Groq API request

POST /openai/v1/chat/completionsbash

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer ${GROQ_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b-versatile",
    "messages": [{"role": "user", "content": "Explain fast inference"}]
  }'

How to authenticate with the Groq API

Aspect	Detail	Notes
Auth	Bearer GROQ_API_KEY	export GROQ_API_KEY=... ; create in the console
Base URL	https://api.groq.com/openai/v1	OpenAI-compatible chat completions
Official SDK	groq (Python: pip install groq; JS)	client.chat.completions.create
Third-party SDKs	Vercel AI SDK (@ai-sdk/groq), LiteLLM, LangChain	Drop-in providers
Models	llama-3.3-70b-versatile, gpt-oss, Qwen, Llama 4 Scout	11 open-weight models, all 131k context
OpenAI migration	Reuse OpenAI SDK by changing base_url + key	Minimal code change
Drop-in role	Fast, cheap backend behind existing apps	Swap provider, keep code

Groq core endpoints and features

Capability	How	Purpose
Chat completions	POST /openai/v1/chat/completions	Text generation (groq SDK or OpenAI SDK)
Streaming	stream=True	Stream tokens at Groq's high output speed
Tool use	tools[] (function calling)	Call functions; supported on all models
Structured outputs	JSON mode / schema	Schema-conforming JSON, all models
Prompt caching	Prompt Caching	Cut cost and latency on repeated context
Speech	Speech to Text, Text to Speech	Audio transcription and synthesis
Compound (agentic)	Compound systems	Built-in agentic AI with tools
Batch + LoRA	Batch Processing, LoRA Inference	Bulk jobs and fine-tuned adapters

Rate limits, tiers and rules

Aspect	Detail	Notes
Rate limits	Per-model rate limits on the API	See Rate Limits doc; raise with tiers
Service tiers	On-demand, Performance Tier, Flex, Batch	Trade latency for cost / scale
Context window	131k tokens (all models)	Uniform across the catalog
Spend limits	Account spend limits + per-project controls	Projects, model permissions
OpenAI-compat caveats	Some OpenAI params differ	See OpenAI Compatibility doc
Error handling	Documented error codes	Retry / backoff guidance
Batch discount	Batch Processing for async bulk jobs	Lower price, higher latency

SDKs, production and developer fit

Aspect	Detail	Notes
SDKs	groq (Python/JS), OpenAI SDK, Vercel AI SDK, LiteLLM, LangChain	Pick by stack
Production checklist	Production Checklist + Optimizing Latency	Ship-readiness guidance
Observability	Prometheus Metrics	Export GroqCloud metrics
Reasoning + moderation	Reasoning, Content Moderation endpoints	Extended thinking + safety
OCR / vision	OCR and Image Recognition	Document and image input
Templates / cookbook	Templates + Groq API Cookbook	Starter projects
API pricing	$0.05-$0.84 / 1M blended by model	Independent cross-reference

Groq API FAQ

How do I authenticate with the Groq API?

Create an API key in the GroqCloud console and set it as the GROQ_API_KEY environment variable, then send it as a bearer token. The simplest path is the official groq SDK, through pip install groq, then client.chat.completions.create with a model like llama-3.3-70b-versatile. Because the API is OpenAI-compatible at https://api.groq.com/openai/v1, you can also use the OpenAI SDK by swapping its base URL and key.

Is the Groq API OpenAI-compatible?

Yes. Groq exposes an OpenAI-compatible API, so you can reuse existing OpenAI SDK integrations by changing the base URL to https://api.groq.com/openai/v1 and the key, with minimal code change. Some OpenAI parameters differ, so review the OpenAI Compatibility documentation before migrating. The Vercel AI SDK, through @ai-sdk/groq, LiteLLM and LangChain also have Groq providers.

What can the Groq API do beyond chat?

GroqCloud covers streaming, tool use on all models, JSON and structured outputs, prompt caching, speech-to-text and text-to-speech, OCR and image recognition, reasoning models, content moderation, and an agentic Compound system. It also adds Batch Processing for bulk jobs and LoRA inference for fine-tuned adapters, all on the OpenAI-compatible surface.

What are the Groq API rate limits and tiers?

Usage is shaped by per-model rate limits and service tiers: on-demand by default, a Performance Tier, Flex Processing for discounted work that can tolerate latency, and Batch Processing for bulk async jobs. Accounts have spend limits and per-project model permissions, and the console exposes a production checklist, latency-optimization guidance and Prometheus metrics for production use.

Which Groq model should I call?

Pick by job. Small gpt-oss models give raw speed, over 900 tokens a second. Llama 3.1 8B is the cheapest fast option, at $0.05 per 1M. Llama 3.3 70B offers stronger quality at around 300 tokens a second, and the Qwen3.6 27B reasoning model has the highest intelligence on Groq. All 11 models run at 131k context with function calling and JSON mode, so switching is just a model-string change.

Sources & verification

Verified by ComparEdgeMethod: Vendor docs, official pages, and selected independent sources

Source	What was checked	Last checked
Groq Official	Official product page	July 10, 2026
Artificial Analysis	Independent reference	July 10, 2026
Groq Openai	Openai	July 10, 2026
Groq Quickstart	Quickstart	July 10, 2026

Every fact on this Groq page is tied to a named source and a verification date. Freshness-sensitive figures trace to the sources above; verify against the vendor before relying on them.