{
      "api_key": "YOUR_KEY",
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "prompt": "Explain recursion in one paragraph.",
      "temperature": 0.7,
      "top_k": 40,
      "top_p": 0.9,
      "max_tokens": 256,
      "repeat_penalty": 1.1,
      "stop": []
    }

Response (200):

{
      "response": "Recursion is a technique where a function calls itself...",
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "instance_id": "abc123",
      "prompt_tokens": 12,
      "completion_tokens": 87,
      "tokens_per_second": 14.3,
      "latency_ms": 6083.2
    }

Error responses include a detail or error field with a human-readable message.

{"error": {"message": "Invalid API key", "type": "authentication_error", "code": 401}}

{
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user",   "content": "What is recursion?"}
      ],
      "temperature": 0.7,
      "max_tokens": 256
    }

Response (200):

{
      "id": "chatcmpl-abc123",
      "object": "chat.completion",
      "created": 1719000000,
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "choices": [{
        "index": 0,
        "message": {"role": "assistant", "content": "Recursion is..."},
        "finish_reason": "stop"
      }],
      "usage": {"prompt_tokens": 25, "completion_tokens": 87, "total_tokens": 112}
    }

Streaming: Set "stream": true in the request body. The response uses Server-Sent Events (SSE):

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"token"},"finish_reason":null}]}

    data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}

    data: [DONE]

Vision (image input)

For vision-capable models (check "vision_capable" on GET /v1/models), "content" can be an array mixing text and image parts instead of a plain string:

{
      "model": "MODEL_NAME",
      "messages": [
        {"role": "user", "content": [
          {"type": "text", "text": "What's in this image?"},
          {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo..."}}
        ]}
      ],
      "max_tokens": 200
    }

Up to 4 images per request, 8MB each (raw, before decoding). Images are automatically downscaled to 768px on the longer side and count toward your token quota.

"image_url.url" accepts either a data: URI (base64) or a public http(s) URL — localhost/private addresses are rejected.

Documents (file input)

Works on any model, not just vision-capable ones — "content" can include a "file" part whose text is extracted and merged into the prompt before the model ever sees it:

{
      "model": "MODEL_NAME",
      "messages": [
        {"role": "user", "content": [
          {"type": "text", "text": "Summarize this document."},
          {"type": "file", "file": {"filename": "report.pdf", "data": "data:application/pdf;base64,JVBERi0xLjQK..."}}
        ]}
      ],
      "max_tokens": 200
    }

Up to 5 files per request, 15MB each (raw, before decoding). Any file with extractable text is accepted (PDF, ZIP, Office docs, source/config files, etc.) — images, audio, video and other non-text formats are rejected. Extracted text is capped at 40,000 characters per file (truncated beyond that) and counts as ordinary prompt text toward your token quota.

"file.data" must be a data: URI (base64) — unlike image_url, remote http(s) URLs are not accepted for file uploads.

A ZIP is unpacked and every supported file inside it (including nested documents) is converted and concatenated.

{
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "prompt": "In one sentence, what is a large language model?",
      "max_tokens": 80
    }

Response (200):

{
      "id": "cmpl-abc123",
      "object": "text_completion",
      "created": 1719000000,
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "choices": [{"text": "A large language model is...", "index": 0, "finish_reason": "stop"}],
      "usage": {"prompt_tokens": 14, "completion_tokens": 18, "total_tokens": 32}
    }

{
      "api_key": "YOUR_KEY",
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "prompt": "Explain recursion in one paragraph.",
      "mode": "async",
      "key": "BASE64_32_BYTE_KEY"
    }

Response (200) — tracking ID, no result yet:

{
      "mode": "async",
      "tracking_id": "10de8f98-88b9-4138-9443-dc14da61150d",
      "method": "xchacha20",
      "nonce": "BASE64_24_BYTE_NONCE"
    }

If the worker falls back from sync to async on its own (no mode was requested), the response also includes a worker-generated "key" field, since you have no other way to obtain it.

If webhook_url was supplied, the worker POSTs this same payload to it once the job finishes (result isn't deleted by the webhook delivery — /api/async_result below still works as a fallback):

{"status": "ready", "tracking_id": "10de8f98-88b9-4138-9443-dc14da61150d", "result": "BASE64_CIPHERTEXT"}

GET BASE_URL/api/async_result?tracking_id=10de8f98-88b9-4138-9443-dc14da61150d&api_key=YOUR_KEY

Response (200) — still generating:

{"status": "pending"}

Response (200) — result ready:

{"status": "ready", "result": "BASE64_CIPHERTEXT"}

"result" is base64-encoded ciphertext — decrypt it yourself with XChaCha20-Poly1305 using the key and nonce from the original request/response. The server never sees the plaintext or stores the key.

Response (404) — unknown tracking_id, expired, or already consumed:

{"error": {"message": "Unknown tracking_id, expired, or its result was already consumed — stop polling.", "type": "not_found_error", "code": 404}}

"pending" only ever means "still generating." A 404 means the tracking_id doesn't exist, expired, or its result was already read by another poll — you will never get a 200 for it again, so stop polling instead of looping forever.

{
      "object": "list",
      "data": [{"id": "Qwen2.5-1.5B-Instruct-Q2_K", "object": "model",
                "created": 1719000000, "owned_by": "5cbot_llm_server",
                "avg_tokens_per_second": 18.2, "vision_capable": false}]
    }

avg_tokens_per_second is a 30-day rolling average (errors excluded). Omitted for models with no benchmark data yet. Use it to estimate latency before sending a request.

vision_capable indicates whether the model accepts image content-parts — see Vision (image input) above.

{
      "instance_id": "abc123",
      "model": "Qwen2.5-1.5B-Instruct-Q2_K",
      "status": "ready",
      "host": "127.0.0.1", "port": 8001, "pid": 12345,
      "active_users": 2,
      "ram_used_mb": 1200,
      "cpu_percent": 45.2,
      "uptime_seconds": 3600,
      "last_activity": "2025-06-17T12:00:00+00:00"
    }