Skip to content
sobolewski.proAI today, every eveningSubscribe

Once a day. Unsubscribe with one click. Privacy policy

Inkling 2 Mini: the open 26B model that runs on a laptop

Inkling 2 Mini from Thinking Machines is an open model with 26.4B parameters that takes about 20 GB and runs on a Mac with 32 GB. Here is what it can do.

4 min read

This article in other languages:Polski

In this article
  1. What’s new here for AI?
  2. How does it score on benchmarks?
  3. What should you watch out for?
  4. How to try Inkling 2 Mini
  5. FAQ

On November 20, Thinking Machines Lab released Inkling 2 Mini, a mixture-of-experts (MoE) model with 26.4B parameters whose Apache 2.0 weights are already on Hugging Face. The whole checkpoint takes about 20 GB and, according to the announcement, runs on a Mac with 32 GB of memory.

What’s new here for AI?

Two weeks ago the same company showed Inkling 2 Small with 594B parameters, a model for a server with several GPUs. Mini is more than twenty times smaller and was built for the hardware that sits on a desk.

The most interesting part is how it’s put together. Inkling 2 Mini is a “looped transformer”: it has 24 layers in four blocks of six, and the two middle blocks run three times for every token. So 24 layers of weights turn into 48 passes. Each token activates 3.4B parameters, or 5.4B once you count the repeats.

The idea itself isn’t new; the announcement points back to the Universal Transformers paper from 2018. What Thinking Machines did check is which layers are worth looping: the middle twelve beat the last twelve on 27 of 28 evals. According to the company, the model “thinks” more per token this way and reaches an answer using fewer of them.

The second thing is precision. Most of the weights were trained in the 4-bit NVFP4 format (the last four layers in MXFP8) and were published in that same form. Usually what we run locally is a quantized copy of a model the lab measured at full precision. Here the official checkpoint is the thing that was trained.

There’s also a demo that’s fun to watch: the company’s researchers took the model on a plane, where on a MacBook Pro (M5 Pro, 48 GB) in airplane mode it built a playable flight simulator in a few minutes.

How does it score on benchmarks?

A few results from the announcement:

  1. DeepSWE v1.1: 48.2%
  2. SWE-bench Verified (bash only): 79.2%
  3. Vals Index v2.2: 46.5%
  4. Artificial Analysis Intelligence Index v5.0: 20.1

These are Thinking Machines’ own measurements, so let’s keep some distance. In the Hacker News thread (over 560 points in three hours) the enthusiasm is mixed with first tests that went both ways. One commenter described the Unsloth Q4 build spending 19,200 thinking tokens on a simple WordPress plugin, the entire limit they had set, without finishing the task.

What should you watch out for?

First, the model’s knowledge. The announcement says the loop improves reasoning but doesn’t increase the number of facts stored in the weights, because that depends mostly on the total parameter count. Inkling 2 Mini is supposed to make up for the gaps with search and tools (offline, as in the plane demo, there is no search engine).

Second, chat. The company admits this isn’t the strongest small model for multi-turn conversations or for instructions with many constraints: 50.0% on MultiChallenge and 33.4% on ComplexConstraints v1.1.

Third, oversight of reasoning. A looped model can “think” for longer without writing down its chain of thought (CoT), and reading the CoT is exactly how we check what a model is doing. Thinking Machines measured this on a separate research variant with a variable number of passes, which it also released. The tasks this variant solves without CoT grew from 16 to 37 seconds of human work when going from one loop to three, and the CoT got shorter but stayed readable.

The company’s conclusion from this preliminary analysis is that “at this scale” there is no meaningful drop in monitorability. The question about larger looped models stays open, and it’s the one that keeps coming back in the comments.

How to try Inkling 2 Mini

  1. On a Mac with Ollama. ollama run inkling-2-mini is enough, but the model’s Ollama page currently requires a pre-release version of the app and a Mac with at least 32 GB of memory.
  2. In the browser. You can chat with the model in the Tinker Playground. Through the Tinker API, output costs $0.50 per million tokens and input tokens $0.15 (pricing), half the Inkling 2 Small rates. The context window goes up to 1M tokens.
  3. On other hardware. The model card lists a single NVIDIA DGX Spark, and for other GPUs recommends at least 32 GB of VRAM. There are also GGUF builds from Unsloth for llama.cpp.

The first speeds reported in the comments are about 52 tokens per second on an RTX 5090 and about 20 on a MacBook Pro with an M1 Max and 64 GB (4-bit build). Those numbers come from the first hours, when llama.cpp support was only just landing.

If this is the first model you’re running on your own machine, I covered Ollama and LM Studio step by step in the guide on how to run AI locally.

FAQ

Is Inkling 2 Mini free?

The weights are under the Apache 2.0 license and can be downloaded from Hugging Face. Locally you pay only in electricity (and wear on your hardware), and in Tinker according to the price list.

What hardware does Inkling 2 Mini need?

According to the model card, an Apple silicon Mac with at least 32 GB of memory or a single NVIDIA DGX Spark. For other GPUs, 32 GB of VRAM is recommended.

How is Inkling 2 Mini different from Inkling 2 Small?

Small has 594B parameters (17B active) and needs a server. Mini has 26.4B, looped middle layers, and fits in 20 GB.

  • AI
  • open-weight
  • AI models

AI today, every evening

The day's biggest AI news and new posts from the blog, every evening. Short, with sources.

Once a day. Unsubscribe with one click. Privacy policy