Meet Bonsai: The First 27B AI Model That Fits on Your Phone - Decrypt

Meet Bonsai: The First 27B AI Model That Fits on Your Phone

I models eat up a lot of memory. A 27-billion-parameter AI model, considered medium-sized by industry standards, needs roughly 54 GB of memory to run on half precision. Most laptops can't hold that. Some desktop rigs can't either.

Earlier this week, PrismML released one at 3.9 GB—small enough to fit on an iPhone.

Parameters are the number of dials and tweaks a model can handle. The more parameters, the denser and more capable a model is.

Bonsai 27B is the first 27B-class model to clear the memory ceiling of a consumer smartphone, running at 11 tokens per second on an iPhone 17 Pro Max. (Tokens are the basic unit of information that AI models can handle and produce.) The ternary variant, at 5.9 GB, hits around 26 tokens per second on an M5 Pro laptop. Both are free under Apache 2.0.

The compression method, built on Caltech intellectual property, reduces each model weight from 16 bits of floating-point precision to a single sign—+1 or -1 in the binary build, one of three values in the ternary. Each group of 128 weights shares a 16-bit scaling factor, landing the binary variant at 1.125 bits per weight: 14 times smaller than the full-precision original. The ternary model adds a zero state for slightly more expressive power and settles at 1.71 bits.

In easier terms, this means a ternary AI model uses only three settings for each internal value—negative, zero, or positive—while a standard AI can choose from about 65,000 settings.

PrismML did that without losing much of the output quality.

What makes this different from conventional "low-bit" models is that nothing gets a higher-precision escape hatch: embeddings, attention, and the full language model head are all compressed end-to-end. Most quantized builds keep certain sensitive layers at full precision, which ends up increasing their size as a tradeoff for better quality. Bonsai doesn't play that game.

This is the second major release in the family. In March, PrismML shipped Bonsai 8B , a 1.15 GB model that proved the 1-bit architecture could survive at 8 billion parameters without its reasoning collapsing. The jump to 27 billion is where the stakes change—that scale is where sustained chain-of-thought reasoning, reliable tool use, and multi-step agentic behavior actually emerge consistently—the things smaller models still fumble.

Across 15 benchmarks evaluated in thinking mode on NVIDIA H100 GPUs—spanning knowledge, math, coding, and tool use—Ternary Bonsai 27B averages 80.49, or 94.6% of the full-precision model. The 1-bit variant hits 76.11.

Overall, on benchmarks, the models perform much better than Gemma 4 or Qwen 3.6 in terms of how much potential they offer for their size.

The models are pretty good for what they offer, and considering how little resources they require, they take small hardware (smartphones and lower end PCs) to another level in terms of capabilities. AIME25 and AIME26, modeled on the American Invitational Mathematics Examination, come in 93.7% for Ternary Bonsai 27B versus 95.3% for the much bigger Qwen 3.6B. Bonsai scores 86 points in codig vs 88 for Qwen 3.6 and 77% on general knowledge vs 83 for Qwen 3.6.

The model also uses a hybrid attention backbone where roughly 75% of the layers are linear rather than full quadratic attention. That architecture is what makes a 262K-token context window practical on-device—something a standard attention stack would make prohibitively expensive on phone hardware.

We ran Bonsai 27B ourselves. Coding takes iteration: single-shot prompts won't compete with cloud frontier models. Being local and free makes that irrelevant. For our Zombie Type game—a first-person typing-horror browser game—two vibe coding rounds produced clean collision detection, proper scoring logic, and graphics that held together. The model grasps structure early; the second pass refines rather than rebuilds.

Interestingly enough, some models (like the skeletons) looked more elaborate than the ones from GPT 5.6 Sol. It doesn’t mean it’s better by any means, just that on this task it produced a cute skeleton whereas the AI king made a poorer stylistic choice.

The game is available for testing here .

Creative writing is a more qualified story, and the criteria is more subjective.

Roughly speaking, the results aren't particularly imaginative if you have a zero-shot prompt in mind.

That said, Bonsai produces stories with consistent internal logic, pacing, and arc—better, or on par with Claude Haiku or even Sonnet on lower effort on comparable prompts. For a model that runs entirely on your own hardware with no API costs, that's a lot to say.

The story it created can be found in our Github repository .

PrismML also ships a DSpark speculative decoding layer alongside the model—a lightweight drafter that proposes blocks of candidate tokens, which the main model verifies in a single forward pass rather than generating token-by-token. On an H100 that adds a 1.37x throughput boost with no change in output quality, since verification preserves the exact output distribution. On Apple Silicon it's not yet enabled by default, but for GPU serving it's a real gain.

Apple's interest adds a commercial dimension. PrismML CEO Babak Hassibi confirmed to CNBC that the company is in early talks with Apple, which is evaluating the compression technology for potential on-device use.

Hassibi said a compressed Gemma model is next in the pipeline, followed by larger frontier models; 1-bit Bonsai 27B is available for free download now under Apache 2.0. If you need a primer on running models like this locally, check out our guide .

Your gateway into the world of Web3

The latest news, articles, and resources, sent to your inbox weekly.

© A next-generation media company. 2026 Decrypt Media, Inc.

新闻图片