Register GGUF models
Administrators register open-source instruction-tuned models in GGUF format (≤ 4B parameters) with RAM estimates and approval flags.
VIP Research Group · Athabasca University
Serving open LLMs on self-hosted CPU servers, one worker per model.
Run open-source instruction-tuned models in GGUF format through isolated llama.cpp workers — with per-user permissions, daily token quotas, and an OpenAI-compatible API.
-
One worker per model
Launched with
--port/--modelparameters; RAM guards, port allocation, and idle eviction built in. - Swappable backend All inference goes through a provider abstraction — llama.cpp today, vLLM or Ollama tomorrow.
-
Playground & API
Test prompts with live benchmarks, or integrate via
/v1/chat/completions.
About the Project
5cbot_llm_server is the LLM inference backbone for the VIP Research Group's conversational agent research. Instead of relying on the HTTP handler that llama.cpp provides, the platform embeds the llama.cpp Python package inside its own FastAPI workers — giving the research group full control over authentication, scheduling, quota enforcement, and benchmarking around every inference call.
Each worker instance is started with command-line parameters such as --port, --model, --instance-id, and --host, listens on its own port, and exchanges JSON request and response packages — callable from a Python socket client, a PHP curl call, or any OpenAI-compatible SDK. The management dashboard launches and supervises the fleet: it assigns ports, enforces a total-RAM budget, evicts idle instances, and lets administrators decide which user can use, start, or stop each model.
Launch isolated workers
The scheduler starts one Python worker per instance with --port/--model parameters; several instances serve different models on different ports concurrently.
Serve JSON requests
Workers answer JSON packages on /generate and the OpenAI-compatible /v1/chat/completions, with per-request benchmark logging (tokens/s, latency, RAM, CPU).
Swap the backend anytime
Inference flows through an LLMProvider abstraction, so llama.cpp can be replaced by vLLM, Ollama, or another engine whenever the research needs it.
How to Access
Users
Sign in, start a model you have permission for, and chat with it in the playground with live benchmarks.
Administrators
Manage users, model permissions, daily quotas, running instances, and platform resource limits.
Developers
Create an API key and call the JSON or OpenAI-compatible endpoints from Python, PHP, or any HTTP client.
API Documentation
Full endpoint reference with copy-paste code examples in cURL, Python, JavaScript, TypeScript, PHP, and Dart.
Terms of Use
The VIP Research Group is led by Prof. Maiga Chang at the School of Computing and Information Systems, Athabasca University. This LLM server is one of the research group's works, with follow-up research planned for future projects.
Almost all of Prof. Chang's works are open access or open source. This service currently runs on a self-sponsored server, like other research projects listed among Prof. Chang's advanced projects. It is intended to remain online, improving, and accessible as long as operating costs are affordable and covered.
If access volume becomes too high, or if a business or commercial institution uses the service for profit, the terms may change to include donations, personal, academic, or business licensing, or subscription models.
About Us
Our Mission
We build open, self-hosted AI infrastructure for education research — language model services, educational chatbots, and the platforms that let researchers run real-course experiments on affordable hardware.
Our Supervisor
Dr. Maiga Chang is a Full Professor in the School of Computing and Information Systems at Athabasca University, Canada, and leads the VIP Research Group's projects in AI for education.
Research Goal
This project investigates how multiple quantized open-source LLMs can be served reliably from CPU-only servers — with fair multi-tenant scheduling, quota enforcement, and backends that remain swappable as inference engines evolve.
Our Team
Prof. Maiga Chang
Supervisor
Dr. Maiga Chang is a Full Professor in the School of Computing and Information Systems at Athabasca University, Canada. He leads the VIP Research Group, whose projects span educational chatbots, natural language processing services, and game-based learning.
Chater MARZOUGUI
Lead Developer · 2026 - Present
Chater Marzougui is a final-year engineering student developing 5cbot_llm_server as his graduation project. He designed and built the multi-tenant worker platform, the llama.cpp provider abstraction, and the management dashboard.
Publications
- Related work Supun DE SILVA and Maiga CHANG. (2025). ChatbotLLM - Training Educational Chatbots on the Materials Uploaded by Teachers. In: 13th International Conference on Information and Education Technology (ICIET 2025), Fukuyama, Japan, April 18-20, 2025. (IEEE) DOI
- Related work Fuzheng ZHAO, Zibusiso MAFAITI, Yongjin LI, and Maiga CHANG. (2026). Mobile-Accessible Seamless Agent (MASA): Using ChatbotLLM for Grounding Chatbots in Teacher-Provided Materials. In: APSCE 4th International Conference on Artificial Intelligence, Metaverse and Artificial Companions in Education and Society (AI-MetaACES 2026), Kumamoto, Japan, June 25-27, 2026 (accepted)
Frequently Asked Questions
What is 5cbot_llm_server?
5cbot_llm_server is a multi-tenant LLM inference platform. It launches one isolated Python worker per running model — each loading an open-source instruction-tuned model in GGUF format through the llama.cpp package — and puts a PHP management dashboard in front for accounts, permissions, quotas, and a prompt playground.
How do workers run multiple models at the same time?
Every model instance is a separate worker process started with its own parameters (--port, --model, --instance-id, --host). Workers listen on different ports, exchange JSON request and response packages, and register themselves in a shared running-instances registry that the dashboard's scheduler uses for port allocation, RAM guards, and idle eviction.
Why llama.cpp, and can it be swapped out?
Workers use the llama.cpp Python package (not its bundled HTTP handler) so the platform fully controls authentication, quotas, and scheduling. All inference goes through a provider abstraction, so llama.cpp can be switched to another backend such as vLLM or Ollama without changing the rest of the system.
Does it need a GPU?
No. The platform targets CPU-only servers. Models up to 4B parameters in quantized GGUF format run on a typical 24 GB RAM machine, and the scheduler enforces a total-RAM budget so concurrent instances never overcommit the host.
Can other applications call it programmatically?
Yes. Each worker exposes a JSON API including an OpenAI-compatible POST /v1/chat/completions endpoint, so existing OpenAI SDK clients work by changing the base URL and using an API key issued by the dashboard. PHP clients can use plain curl calls and Python clients a simple socket/HTTP request.
How is access controlled?
Administrators create accounts and grant per-model permissions (use, start, stop) with daily token quotas per user. Every inference request is authenticated with a hashed API key, usage is logged for quota enforcement, and benchmark metrics (tokens/second, latency, RAM, CPU) are recorded per request.
Is the service open access?
Like other VIP Research Group projects, the service is intended to stay online and accessible as long as operating costs are affordable and covered. If access volume becomes too high or commercial use appears, the terms may change — see the Terms of Use section.