Home › LLM engine
A language-model stack written in dependency-free C.
No PyTorch, no BLAS, no framework — the C standard library and the maths. Training, inference, fine-tuning and a tool-using agent, all hand-written.
What it does
It trains and runs transformer language models. Two architectures are implemented and both train: a classic GPT-2 style network, and a modern LLaMA-family one with RMSNorm, rotary position embeddings, grouped-query attention and SwiGLU. Every backward pass is derived by hand and checked numerically against finite differences.
It loads real published weights, runs them with a KV cache and quantised int8 tensors, fine-tunes them with LoRA adapters, and then uses the result as the brain of a tool-using coding agent — parse, approve, execute, observe — with no API and no network.
Why it is interesting
- The tokeniser is written in C, matching the reference byte-pair encoders exactly, so there is no Python anywhere at runtime.
- The matrix kernels are plain portable C with no intrinsics, and still reach roughly seven to nine times the speed of the naive version through cache blocking alone.
- It cross-compiles to eight targets including WebAssembly, and runs there.
- The agent is fine-tuned by the project's own trainer — the stack trains the model it then runs.
Scope, stated plainly. This is a complete and correct engine, not a competitive model. Trained from scratch on a CPU, the outputs are toy-scale by design. The deliverable is the pipeline — every stage implemented and verified — rather than production quality text.
Where it stands
Working and actively developed. There is a 25-part self-test that gates every change, including numerical gradient checks and a bit-exactness check on the KV cache, and a browser UI for driving training and generation.