Weight-Space Programming: GPT-6 Astra fixed Qwen's weights

On November 2, four researchers from MIT CSAIL (Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab) published a post called “Sparks of Weight-Space Programming”. In it they show that OpenAI’s GPT-6 Astra can read the raw weights of a neural network, work out what it was trained to do, and fix them by hand, with no gradients and no training loop. Within an hour the post had more than 650 points on Hacker News.

What is Weight-Space Programming?

That’s the authors’ name for a setup in which a general-purpose language model, used “as is”, directly reads and writes the weights of another network according to a goal described in plain language. No optimizer like Adam, no gradient descent, and in some experiments not even any feedback from the environment.

As the researchers put it, Astra “uses reasoning to figure out what each part of the network computes, or what it should compute”, and changes the weights accordingly. The team describes eight experiments in three groups: reading weights, fixing weights, and writing them from scratch.

What did Astra read from the weights alone?

1. It identified a game from 17,253 numbers. The researchers used PPO to train four networks with identical architecture on Breakout-, Pong-, Pac-Man- and Freeway-style games. Astra got only the weights as text, with no list of games, and named the correct game for all four networks.
2. It read a shape out of a diffusion model. Without running the network, it said whether the model draws a circle, a square or a triangle. Interestingly, it drew them as outlines, even though the models generate filled shapes, which according to the authors shows it was reading the geometry of the weights rather than computing the output.
3. It identified the language of a small language model (7.2M parameters) using only the middle transformer blocks, without the input and output layers. It got Python, romaji and English right in every trial, and pointed to exactly the attention heads that copy earlier text (so-called “induction heads”).

How did GPT-6 Astra fix Qwen?

This is the part everyone is talking about. Astra got a Python session with the weights of Qwen3.5-0.8B (no GPU), training tasks from GSM8K and MATH, and a budget of 10, 25 or 50 model runs. The goal: better math, without training.

Astra noticed a problem that Qwen’s own model card warns about: the model falls into thinking loops and never reaches an answer. It traced the loops to a single direction written by MLP layers 15 and 19, then subtracted it from 64 output columns in each of them. That’s 131,072 of 0.8 billion weights, or 0.016%.

Qwen3.5-0.8B	MATH-500	GSM8K	IFEval
Base	7.9%	13.8%	21.0%
After the edit (50 runs)	48.1%	67.1%	62.3%

The share of MATH-500 answers that hit the 32,768-token limit fell from 93% to 33.1%. With 10 and 25 runs the gain was much smaller (MATH-500 at 12.3% and 14.1% respectively).

Can Astra write a network from scratch?

According to the post, yes, at least a small one. In the MinAtar games, Astra got the rules, the game’s source code and the network layout (all weights set to zero), and then wrote out the weights as text in a single response. No data, no tools, and not a single game played before evaluation.

1. In Breakout, Astra’s network scored 517.4 points, versus 25.3 for PPO trained for 500M steps.
2. In Asterix, 53.2 points versus 55.2 for PPO after 500M steps.
3. In Seaquest, 53.1 points versus 88.9.

In Breakout, Astra programmed the first layer to calculate where the ball would land and the second to move the paddle there. A 9×9 Go network with 1.48M weights was built the same way: with 50 runs it beat a PPO network of the same size trained on 20M games.

What should you keep in mind?

First, the failures the authors describe themselves. From the weights alone, Astra couldn’t tell a “sycophantic” version of Gemma 3 270M from an honest one (it only succeeded with access to attention patterns). And Gemma 3 270M, which rarely loops, barely improved on MATH-500 after the edits (6.8% before, 7.7% at best after). As the researchers write, editing weights can remove an obstacle that was hiding a skill, but it didn’t add a skill the model didn’t have.

Second, scale. A very large model was fixing a 0.8B-parameter network, and HN commenters point out that removing a single direction from the weights is a known technique (so-called “abliteration”). Nobody has independently reproduced the results yet, but the full logs of every experiment are public.

Third, direction. The authors explicitly mention recursive self-improvement: a model that improves its own weights, or the weights of a newer model. For now it’s a vision, but the thread closely resembles Recursive Flint and the proof from OpenAI’s internal model, where AI is increasingly doing research work.

FAQ

What is Weight-Space Programming?

It’s an approach in which a language model directly reads and writes the weights of another neural network according to a goal described in words, with no gradient descent and no training loop. The name was proposed by MIT CSAIL researchers in a post from November 2, 2026.

Is GPT-6 Astra available?

Yes, GPT-6 Astra is the top tier in OpenAI’s model family (from Luna, through Terra and Sol, to Astra). The tools from the experiments haven’t been released as a ready product: the authors say they are only now building a benchmark and a harness for Weight-Space Programming.

Can a large model be fixed this way?

We don’t know yet. The largest edited model had 0.8 billion parameters, and the authors themselves write that the capabilities are “still limited”, mainly by context length and the design of the harness.