A 27B model that runs on a phone used to be a research talking point. PrismML’s Bonsai 27B makes it real. The 1-bit variant is 5.9GB. That is the headline number, and it is backed by a benchmark comparison that is worth walking through in detail.
What Bonsai 27B Actually Is
Bonsai 27B is based on Qwen 3.6 27B, a known strong performer in the 27B class. PrismML did not redesign the architecture. They changed how the weights are stored. Ternary weights from {-1, 0, +1} with FP16 group-wise scaling give 1.71 effective bits per weight. The 16-bit original needs 54GB. Bonsai 1-bit needs 5.9GB. The ternary variant sits at 18GB for laptops with discrete GPUs.
9x compression. That is the underlying number everything else rests on.
The Density Numbers
PrismML evaluates this on intelligence density: benchmark score divided by model size in GB. Full-precision Qwen 3.6 27B scores about 0.05 per GB. Bonsai 1-bit scores 0.53 per GB. Ten times the density.
For local inference, that is a different cost structure. No API calls. No per-token billing. Data stays on device. If your use case requires privacy, offline operation, or sub-100ms response, the math shifts in favor of local in a way it did not twelve months ago.
The Benchmark Gaps
Ternary Bonsai 27B vs full-precision Qwen 3.6 27B:
– Math: 93.4 vs 95.3, gap 1.9
– Coding: 86.0 vs 88.7, gap 2.7
– Agentic/Tool-calling: 74.0 vs 80.0, gap 6.0
– Instruction following: 71.8 vs 78.4, gap 6.6
– Knowledge/STEM: 77.0 vs 83.1, gap 6.1
– Vision: 65.2 vs 72.6, gap 7.4
– Overall: 80.5 vs 85.0, gap 4.5
Math and coding show the smallest gaps. Vision and knowledge show the widest. The overall gap is 4.5 points. For tasks that do not push the edges of vision or knowledge retrieval, the practical difference in everyday use is small.
What the Model Can Actually Do
Bonsai 27B ships with multi-step reasoning, structured tool calls, vision input, and computer-use agent loops. Demo videos show it running the Hermes agent. These are not features added after quantization. They are part of the core capability set, which makes this more of a production bet than a research demo.
The Tradeoff Is Real
Ternary quantization is lossy. Group-wise FP16 scaling recovers some of what is lost when weights are forced into three discrete values, but not all of it. For knowledge-intensive tasks or vision tasks that need high fidelity, full-precision still wins. For most consumer-facing features, the gap is small enough that users will not notice in practice.
One thing worth flagging: PrismML published a whitepaper on the quantization method. That is not standard in this space. Most quantized releases come with marketing claims and no methodology to audit. Having something to point to and verify is the exception, not the rule.
What This Means for Mobile AI
The 18GB ternary variant is the practical entry point for most developers right now. Laptops with 16GB of RAM can handle it without issues. The 5.9GB 1-bit variant is the milestone, but iOS and Android deployment requires inference stack work that is still maturing.
The 10x density number is the thing I keep coming back to. 54GB to 5.9GB is not incremental compression. It moves the model from a category that requires datacenter hardware to a category that fits in your pocket. Whether the benchmark gaps matter depends on what you are building. For coding and reasoning tasks, which drive most of the actual demand for large models, the gaps are small. For vision-heavy or knowledge-heavy tasks, they are still there and worth accounting for.
PrismML has put out a whitepaper and a set of benchmark numbers. The next step is independent testing. If the density numbers hold up outside PrismML’s own evaluation environment, this is a meaningful step forward for on-device AI.
