unillm / cognisoc
Features How it works Use cases Quickstart Blog Compare FAQ About Contact Docs GitHub
Quickstart

From clone to first tokens.

No Python runtime, no server to stand up. Clone the workspace, run the tests, and generate on CPU. GPU acceleration is on the roadmap; everything here works today.

  1. Clone and type-check

    Grab the workspace and let cargo verify it compiles cleanly.

    git clone https://github.com/cognisoc/unillm.git
cd unillm
cargo check

  2. Run the tests

    The workspace ships 201 passing tests. Running them confirms your toolchain is set up.

    cargo test --workspace

  3. Generate text on CPU

    Generate from TinyLlama. The ~600 MB GGUF downloads on first run; inference runs on CPU today.

    cargo run --bin unillm -p unillm-runtime -- \
  generate --prompt "Explain gravity"

  4. Embed it in your service

    Load weights, build a Model, and call generate() straight from your own Rust — no Python server in the loop.

    let weights = WeightLoader::auto_detect("model.gguf")?;
let model = Model::from_weights(weights, Device::Cpu)?;
let text = model.generate("Hello", 128)?;

What to expect

Inference runs on CPU. LLaMA-family models load from GGUF and generate end to end. The other architectures are unit-tested and validated as real weights come online. See the ROADMAP for an honest snapshot, or the architecture page for how the pieces fit.

Full documentation →