Preview Release

cmgenerate v0.2.0

Experimental, high-performance local LLM inference tooling.

⚠️ Software Disclaimer & Notice of Non-Affiliation

The binaries hosted here are early preview builds maintained in an independent, personal staging directory.

• No Corporate SLA / Warranty: This utility is provided strictly "as-is", without warranties, formal commercial support, or liability for operational downtime, data corruption, or system instability.
• AI Output Notice: Local model responses are non-deterministic. Users bear full responsibility for validating generated outputs and ensuring compliance with applicable regional AI safety, privacy, and licensing frameworks.
• System Compatibility: Requires compatible system libraries, drivers (CUDA/ROCm/Metal where applicable), and architectures. Verify hardware targets before deployment.

For Alpeh Alpha Kolibri-1

    
      $ hf download Aleph-Alpha/Kolibri-1
      $ cmgenerate ~/.cache/huggingface/hub/models--Aleph-Alpha--Kolibri-1/snapshots/e52eb4627d11516b0c01de49210ab5a4e4061444 "Explain wha
t Kolibri from Aleph Alpha is"
      /home/andy/.cache/huggingface/hub/models--Aleph-Alpha--Kolibri-1/snapshots/e52eb4627d11516b0c01de49210ab5a4e4061444 (safetensors)
        architecture: kolibri1
        shards: 32
        tensors: 403 x BF16, 57950 x F8_E4M3
        shape: 50 layers, dim 2560, hidden 512, 48 heads over 4 kv heads, vocab 128000
        experts: 384 per layer of 512 wide, 6 used per token
        rope: base 10000, Neox; context 262144 tokens
        weights: 73.4 GiB mapped in 502.8ms
        kernels: avx512f + vnni + vbmi on 32 threads
        cores: 32 threads over 256 cores of 512, 32 cache domains, 2 memory nodes; unpinned; watching 1903 spins
        kv cache: 127.5 MiB as q8 for 4096 tokens (the model allows 262144, which would want 2.7 GiB)
        vocabulary: 128000 pieces
        chat: ChatML, from the markers in the vocabulary
        sampling: temperature 1.00, top-k 40, top-p 0.90, seed 845449043
        speculation: off

      Explain what Kolibri from Aleph Alpha is
      
      1.  **Analyze the Request:**/home/andy/.cache/huggingface/hub/models--Aleph-Alpha--Kolibri-1/snapshots/e52eb4627d11516b0c01de49210ab5a4e4061444 (safetensors)
        architecture: kolibri1
        shards: 32
        tensors: 403 x BF16, 57950 x F8_E4M3
        shape: 50 layers, dim 2560, hidden 512, 48 heads over 4 kv heads, vocab 128000
        experts: 384 per layer of 512 wide, 6 used per token
        rope: base 10000, Neox; context 262144 tokens
        weights: 73.4 GiB mapped in 502.8ms
        kernels: avx512f + vnni + vbmi on 32 threads
        cores: 32 threads over 256 cores of 512, 32 cache domains, 2 memory nodes; unpinned; watching 1903 spins
        kv cache: 127.5 MiB as q8 for 4096 tokens (the model allows 262144, which would want 2.7 GiB)
        vocabulary: 128000 pieces
        chat: ChatML, from the markers in the vocabulary
        sampling: temperature 1.00, top-k 40, top-p 0.90, seed 845449043
        speculation: off

      Explain what Kolibri from Aleph Alpha is
      
      1.  **Analyze the Request:**
[...]

    
      

Available Binary Builds

Operating System	Architecture	Archive	Action
Linux	aarch64 (ARM64)	cmgenerate-0.2.0-linux-aarch64.tar.gz	Download
Linux	x86_64 (Intel/AMD)	cmgenerate-0.2.0-linux-x86_64.tar.gz	Download
macOS	aarch64 (Apple Silicon)	cmgenerate-0.2.0-macos-aarch64.tar.gz	Download
macOS	x86_64 (Intel)	cmgenerate-0.2.0-macos-x86_64.tar.gz	Download
Windows	aarch64 (ARM64)	cmgenerate-0.2.0-windows-aarch64.zip	Download
Windows	x86_64 (Intel/AMD)	cmgenerate-0.2.0-windows-x86_64.zip	Download