API reference
Chat completions
POST/v1/chat/completions
In the OpenAI format, with and without streaming. The same body and the same SDKs; only the base URL and the API key change.
The same body as OpenAI
Official SDK, LangChain, hand-rolled fetch: if it builds an OpenAI request, it works here.
model- The catalog id, exactly as listed. See Models.
messages- The history:
system,user,assistantandtool. streamtrueto receive it over SSE (Streaming).max_tokens- Response cap. Above your plan's cap, you get 400
max_tokens_exceeds_cap. tools- Function tools, as in OpenAI.
temperature- Sampling, as in OpenAI.
{ "id": "chatcmpl-…", "object": "chat.completion", "model": "gpt-6.2-astra", "choices": [{ "index": 0, "message": { "role": "assistant", "content": "Sure!…" }, "finish_reason": "stop" }], "usage": { "prompt_tokens": 9, "completion_tokens": 38 }}Text arriving in chunks
Pass stream: true and the response comes as Server-Sent Events, chunk by chunk.
: heartbeatdata: {"choices":[{"delta":{"content":"Sure"}}]}data: {"choices":[{"delta":{"content":"! Tell"}}]}: heartbeatdata: {"choices":[{"delta":{"content":" me"}}]}data: {"choices":[{"delta":{},"finish_reason":"stop"}]}data: [DONE]Lines that start with : are heartbeats. Ignore them, and do not close the connection because of them. During long reasoning, you may get only heartbeats for quite a while before the first token.
An error mid-stream arrives as a data: event with the error object, followed by data: [DONE].
One conversation per end user
If your app serves many people, send a stable ID per conversation. Same conversation, reused cache, lower cost. Without it nothing breaks; it just costs more.
- The
X-QG-Conversation-IDheader, with the same value on every turn of the conversation. - Up to 128 printable characters.
- An invalid value is ignored, with no error.
- Without an ID, we use the session headers your client already sends, or the start of the prompt.
client.chat.completions.create( model="gpt-6.2-astra", messages=history, extra_headers={ # stable per end-user conversation "X-QG-Conversation-ID": f"u{user.id}-c{conversation.id}", },)Hold first, charge at the end
- Before forwarding, your balance sets aside a hold based on the request size and
max_tokens. If your available balance does not cover it, the response is 402insufficient_balance. - At the end, you are charged for the tokens the model used, at the model's price. The rest of the hold goes back to your balance.
- If the request fails before the response, the whole hold goes back.
- Bonus balance pays for the models that accept bonus; the others use only topped-up balance (Models).