MetaPatchAI home
AI

AI Meets Retro Games: A World Model on a Raspberry Pi Pico, and a Neural Network Built Inside DOOM

By MetaPatchAI AI · Published · Updated

Oct 30, 2026: Two hobby projects on the Hacker News front page this week flip the usual "AI plays games" idea around. One runs a tiny AI that dreams up playable games on a microcontroller. The other builds a working neural network out of a DOOM level. Both are open source, and both are a fun way to understand how these models actually work.

tinyworldformer: games dreamed on a Pico 2 W

Developer SheepSeb's tinyworldformer runs two small neural networks on a Raspberry Pi Pico 2 W at the same time:

  • A world model that takes the current frame and an action, and predicts the next frame and the reward. In other words, it is the game.
  • An actor network that looks at those generated frames and picks the next action. In other words, it plays the game.

According to the project page, on the chip neither network ever touches the real game: the actor only sees frames the world model invents, and the world model only gets actions from the actor. Press any button and control passes to you, so you can play the "dream" yourself.

The project covers five small games: Breakout, a raycast "Doom-lite," Flappy Bird, Pong and Pac-Man. The world model is a single transformer block applied repeatedly (a "looped" transformer), which the developer chose because reusing weights saves memory. On the Pico it runs two loops to keep the frame rate up. Weights are trained with int8 quantization in mind and compiled into Rust firmware.

The write-up is also honest about failure modes. An early Breakout model drew two balls at once, and an agent trained inside the dream once scored 154 points with 26 bricks left because the world model didn't remove corner bricks when they were hit. That is the same "exploit the simulator" problem big labs face, shown on a board that fits in your palm.

doomformer: a neural network made of voodoo dolls

The second project, doomformer by mottopanikeiku (a Show HN post), compiles an integer neural network into a DOOM map. Classic DOOM "voodoo dolls" do the arithmetic, and light levels and floor heights hold the values. The headline demo reads handwritten digits (MNIST): the marine walks the shape of a digit on the floor, and 1,355 voodoo dolls in the map compute the answer.

The author says every value DOOM computes is checked against a NumPy reference with zero tolerance. A playable "museum" WAD uses 136 dolls and lets you rewire one connection mid-game to watch a 7 turn into a 1. The project says a self-test version runs exactly in GZDoom, LZDoom, UZDoom and ZDoom 2.8.1, using the free Freedoom data files.

Why gamers should care

World models like Google DeepMind's Genie 4 are the high-end version of tinyworldformer's idea: an AI that generates the game frame by frame. Seeing it at toy scale makes the trade-offs obvious: small errors compound, and an AI player will find and abuse every mistake in the simulation. Doomformer is pure fun, and a reminder that the DOOM community will run anything inside DOOM.

What's confirmed / not confirmed

Confirmed (from each project's GitHub page): the hardware, the five games, the two-network design and the failure examples for tinyworldformer; the voodoo-doll design, doll counts, zero-tolerance checks and supported source ports for doomformer.

Not confirmed: frame rates on the Pico and accuracy figures beyond what the authors publish. We have not built either project ourselves.

Sources

Up: AI