Bytes Over the Wire: How agar.io Stays Fast

1 The Network tab is suspiciously quiet

Open agar.io, start a match, and pop devtools before you die. Network tab, filter to XHR and Fetch — the /api/whatever handing back JSON that basically every site on earth leans on — and watch a game with thousands of moving things in it hand you almost nothing. No polling, no endpoint returning a tidy blob of state. For something this alive, the panel is eerily still.

Everything is hiding in the one row you’d normally scroll past: the WebSocket. Open it and there are no requests and responses at all, just frames, and the frames aren’t text. Little columns of hex that mean nothing until you know the rules.

That silence is the interesting part. The circles are easy — eat the smaller ones, run from the bigger ones. The machine you never look at is the good one. Every blob on your screen is a person on their own connection somewhere in the world, and all those screens have to agree, constantly, on where everything is — in those frames, and nowhere else.

2 Both ends were melting to speak English

Years ago I built one of these. Not agar.io — a smaller cousin, my own real-time .io game, many evenings poured into a plane full of moving dots. The server was Tornado, Python’s async framework, a natural home for a few hundred long-lived connections that each say almost nothing but say it constantly. Then a fire took it. Repo gone, no backup, no drama. What stayed was the shape of the problem, which turns out to be the part worth keeping.

It spoke JSON, because JSON is what you reach for. Every tick the server serialized each player’s view into a pretty object graph — {"id": 40021, "x": 1234, "y": 9981, "r": 52, "name": "..."} per blob — and json.dumps’d it down the socket. A crowded snapshot ran around 2.4 KB per player per tick. At 20 ticks a second, with the room filling, the event loop started sliding behind around 150 players. The motion got gummy, then it stuttered.

Then I did the thing you have to do before you fix anything, which is prove where the cost lives. Not the network; the box had bandwidth to spare. CPU, on both ends. json.dumps chewed the main loop on the server, and in the browser profile JSON.parse sat right in the frame path, fighting the 60fps render loop for the same milliseconds. I was making two machines speak a human language to each other thousands of times a second.

So I stopped sending the words. Look at where that snapshot spends itself: for every blob on the screen it spells out x, then y, then r, then name, in text, twenty times a second, to a program that already knows what it asked for. The field names were most of the packet. The fix was to agree on the order ahead of time instead — this many bytes for the id, these two for the horizontal position, these two for the vertical — and then send only the numbers, back to back, nothing between them. One marker at the front saying what kind of message this is, and after that nothing on the wire spells out x or name ever again. The crowded snapshot dropped by roughly an order of magnitude, ~2.4 KB down to under 200 bytes. Same information, same box.

That’s what sent me back to devtools. I’ve been poking at rebuilding this kind of thing lately — the small .io games I keep tinkering with live in bytes-over-the-wire — more out of curiosity than plan, and what I kept circling was what the good version looks like at a scale I never got near. agar.io is the one everybody has already played, so I read its wire.

3 Direction picks the meaning

Every frame, both directions, opens the same way: one uint8 at offset 0, the opcode, saying what kind of message this is. Everything after it is positional — known widths in an order both ends agreed on before the connection existed, every multibyte number little-endian, lowest byte first.

Press space to split and the entire message is one byte:

// the whole split packet
const SPLIT = 17;
ws.send(new Uint8Array([SPLIT]));   // opcode, no payload. that's the message.

Eject mass is opcode 21, one byte. Spectate is 1, one byte. The server already knows who you are from the connection; it doesn’t need you to spell out {"action":"split"}, and it doesn’t need a payload to go with the verb. The actions a panicking player mashes hardest cost the cheapest thing you can put on a socket.

The input you send constantly — where your mouse is — is barely bigger: opcode 16, then X and Y, then a uint32 naming the cell you’re steering. Early versions sent those coordinates as float64, two eight-byte doubles for a mouse position on a bounded map; later versions shrank them to 32- and then 16-bit integers.

The tables are small enough that the two directions reuse numbers, which is the detail that catches people. 16 going up is your mouse; 16 coming down is the world snapshot. 255 going up is a reset-connection carrying a uint32 protocol version; 255 coming down is a compressed packet wrapping a bigger one. Nothing inside the byte tells you which. Direction picks the meaning, and each end is expected to know which side of the wire it’s standing on.

4 Ida y vuelta: the whole conversation

With no field names on the wire, a stale client and a fresh server wouldn’t fail, they’d quietly corrupt each other. So the first thing across is a version handshake, two reset-connection packets carrying uint32 versions, then the nickname as a null-terminated string. Only then does the stream start.

CLIENT                                                      SERVER
  |                                                            |
  |  ws.binaryType = "arraybuffer"                             |
  |  (every frame lands as an ArrayBuffer, never a string)      |
  |                                                            |
  |  --- 255  reset-connection  [u32 protocolVersion] -------> |
  |  --- 254  reset-connection  [u32 clientVersion]   -------> |
  |  --- 0    set-nickname      [null-terminated string] ----> |
  |                                          allocate the player
  |  <-- 64   set-border  [f64 x1][f64 y1][f64 x2][f64 y2] ----|  the map edges
  |  <-- 32   add-node    [u32 cellId] ------------------------|  "this one is yours"
  |                                                            |
  |                                     === world snapshots ===|
  |  <-- 16   update-nodes  [u16 eatCount][(u32,u32) x N]      |
  |             [node record][node record]...[u32 0] ----------|  YOUR viewport only
  |  <-- 17   update-position  [f32 x][f32 y][f32 zoom] -------|  where to point the camera
  |                                                            |
  |  --- 16   mouse-move  [X][Y][u32 cellId] ----------------> |  constantly
  |  --- 17   split (1 byte) --------------------------------> |
  |  --- 21   eject (1 byte) --------------------------------> |
  |                                                            |
  |  <-- 255  LZ4 wrapper around { 16 | 64 } ------------------|  only the fat ones
  |  <-- 49   leaderboard  [rows] -----------------------------|  ~1 Hz, nobody's hurry
  |                                                            |
  |  between snapshots the client still draws ~60 fps:          |
  |    remote blobs eased between the last two snapshots,       |
  |    my own blob predicted forward from my own mouse          |
  v                                                            v
 time                                                        time

Everything else here is a zoom-in on one of those arrows.

5 Fourteen bytes for a blob, and none of them say “x”

The fat arrow down is 16, update-nodes, and it resolves deaths before it reports positions: after the opcode comes a uint16 count of the cells just eaten, then that many (eaterId, eatenId) pairs, and only then the survivors. Eating isn’t a radius that happens to reach zero; it’s a destruction list at the head of the packet, so the client retires a blob and credits its mass before it reads where anything is. Then the node records run back to back until a uint32 zero says stop.

  one node record  (16-bit coordinate flavour)
+--------+-------+-------+--------+----+----+----+-------+---------+
|   id   |   x   |   y   | radius |  R |  G |  B | flags |  name?  |
|  u32   |  i16  |  i16  |  u16   | u8 | u8 | u8 |  u8   |  str*   |
+--------+-------+-------+--------+----+----+----+-------+---------+
offset 0     4       6        8     10   11   12    13      14
   4         2       2        2      1    1    1     1    0 or n+1

  * the name rides along ONLY when a bit in `flags` says it's there.
    null-terminated: UTF-16 on protocol <= 5, UTF-8 from 6 on.
    no name flag -> 14 bytes flat  <-- the steady-state case

Fourteen bytes for a blob, and not one of them spells out x. The only variable-length thing is the name, and it shows up only when that flag bit admits there is one: a nick goes out when a client first meets that player’s cell, and after that the bit stays off and the client looks it up from what it already has. A 12-character UTF-8 nick plus its terminator is 13 bytes, almost as big as the record carrying it, and a split player owns a lot of records.

Reading it back is walking an offset:

// walking one node record out of the frame
const dv = new DataView(buf);
let o = 1;                                       // offset 0 was the opcode
const id     = dv.getUint32(o, true); o += 4;    // <-- true = little-endian
const x      = dv.getInt16 (o, true); o += 2;
const y      = dv.getInt16 (o, true); o += 2;
const radius = dv.getUint16(o, true); o += 2;
const r = dv.getUint8(o++), g = dv.getUint8(o++), b = dv.getUint8(o++);
const flags  = dv.getUint8(o++);
// ... name only if (flags & NAME_BIT), then straight into the next record

The offset is the parser. No search, no delimiter, no key lookup, no object graph handed to the garbage collector afterwards. And that , true on every getter is not decoration: DataView defaults to big-endian and this wire is little-endian everywhere. Forget it on one getter and every coordinate comes back byte-swapped. A blob sitting at x = 1234 — 0x04D2, bytes D2 04 — reads back as 0xD204, which as a signed int16 is -11772: thousands of units off the map, in the wrong direction, and the whole field snaps into a thin diagonal smear jammed into a corner. Not a crash. One , true per getter and it unfolds. The most honest kind of bug there is: the bytes are fine, you’re just reading them in the wrong direction.

6 The frames the server never sent

Snapshots arrive far less often than your monitor draws, and the game still looks like a smooth sixty. That gap is a lie the client tells you, and a load-bearing one. It runs two tracks in the same frame: remote blobs get eased between the two most recent snapshots, so they render about one snapshot behind but perfectly smooth, while your own blob can’t afford to lag your mouse and moves the instant you do, locally, then gets nudged back toward the server’s version whenever the next snapshot disagrees.

Notice what that buys the record above: no velocity field anywhere in it. Nothing on the wire says where a blob is heading. The client works out motion by differencing the two positions it already has.

This is the part my old Tornado game got backwards. The motion was jittery, so I sent more snapshots to smooth it, and melted the event loop faster. The answer was the opposite: fewer, tighter snapshots, and let the client invent the frames in between. Smoothness is the client’s job. The server’s job is to be authoritative and quiet.

7 The world is your neighbourhood, not the map

Here’s the decision that was invisible in the Network tab until I noticed what wasn’t in the packet. That update-nodes stream is not the world. It’s your world. The whole map and everyone on it live server-side; each client is handed only the slice around its own cells. Your snapshot stays a few dozen records whether ten people are playing or ten thousand, because it scales with what fits on your screen.

                    +-----------------------------+
                    |           SERVER            |
                    |  the whole map: every cell, |
                    |  every food pellet, virus   |
                    +--------------+--------------+
              query(x, y, viewExtent)   once per tick, per player
        +---------------+----------+----------+---------------+
        v               v          v          v               v
    player A         player B   player C   player D        player E
   one screenful    its slice     ...        ...          its slice
   (~100 records)                                   (bigger blob = bigger slice)

   nobody ever receives the map. everybody receives a neighbourhood.

The loop that does it is unremarkable, which is the point. Roughly what my old Python server did, from memory:

# the shape my old Tornado server used, from memory
def on_message(self, data):
    op = data[0]                      # one byte, offset 0, always
    if op == MOUSE_MOVE:
        x, y, cell_id = struct.unpack_from("<hhI", data, 1)
        self.player.target = (x, y)
    elif op == SPLIT:                 # nothing to unpack
        self.player.split()
    elif op == EJECT:
        self.player.eject()

def tick(self):                       # once per tick, for every player
    for p in self.players:
        near = self.world.query(p.x, p.y, p.view_extent)
        p.send(encode_update_nodes(near))

struct.unpack_from with a format string and an offset — < little-endian, hh the two signed coordinates, I the cell id — and that one line is the whole decoder, because the layout is the parser on both ends. The line that matters is world.query: it decides how big the packet will be, and never asks how many people are online.

Byte packing shrinks a message. Culling decides there’s a small message to send at all, no matter how big the server gets. The record layout is per-game detail. The tower is the engine.

8 Count the bytes, then count them as JSON

Because every field has a known width, you can add a snapshot up exactly instead of guessing. Take a busy screen: 76 player cells plus food and a virus, about 100 records. At 14 bytes each in the steady state that’s ~1.4 KB, plus the opcode, the eaten-pairs list and the terminator. The same picture as compact JSON — {"i":40021,"x":1234,"y":9981,"r":52,"c":1274} is 45 characters, call it ~40 bytes a record after shaving the keys as hard as anyone realistically would — is ~4 KB.

  one busy viewport, ~100 node records
  ------------------------------------------------
  binary update-nodes, steady state     ~1.4 KB
  the same picture as compact JSON      ~4 KB      (~2.9x bigger)
  at 25 snapshots/s:  ~35 KB/s   vs   ~100 KB/s    per client

agar.io compresses on top of that, but selectively: opcode 255 coming down is an LZ4 wrapper, and it only wraps the fat packets, the update-nodes stream and the map border. That selectivity is the design. LZ4 isn’t the strongest compressor available; it’s one of the fastest to decompress, and the client pays that cost inside every frame it draws. A better ratio you have to unpack slowly is a worse deal here.

But size was the cheap half, and not the wall I hit years ago. That was JSON.parse on the hot path: thousands of fields a second, every one competing with the render loop, every parse leaving an object graph for the collector to sweep later as a visible hitch. Text is a lovely language for humans and a quietly expensive one for machines that have to speak it constantly.

9 The pattern under the game

Take the blobs away and the shape is old. The two-track trick — remote entities eased between snapshots, your own predicted ahead of the server — isn’t anyone’s clever idea from 2015: QuakeWorld shipped it in 1996, burying dial-up latency under client-side prediction and delta compression. Valve wrote the modern version down for the rest of us: a Source server, the engine under Counter-Strike and Team Fortress, ships delta-compressed entity snapshots against the last baseline each client acknowledged, then leans on the same predict-and-interpolate client. Blizzard gave a whole GDC talk on Overwatch’s netcode — the same server-authoritative, prediction-heavy spine, around a strict entity-component system. Glenn Fiedler catalogues the three ways to network a simulation, and agar.io sits squarely in the snapshot-interpolation family, which is how you know which corners it’s allowed to cut. The other branch — ship the inputs and run one identical deterministic simulation everywhere — is how Age of Empires fit an army through a modem: “1500 Archers on a 28.8” sent each player’s commands rather than the positions of fifteen hundred units, the same instinct rollback netcode runs in fighting games today, predicting the opponent’s input and rewinding when the guess was wrong.

And it isn’t only games. Leaving field names off the wire is what a schema serializer automates: both ends compile the same Protocol Buffers schema, so the bytes carry values and nothing else, the way that node record carries a name-present bit instead of the word "name". FlatBuffers and Cap’n Proto push it further, reading fields straight out of the buffer with no parse step at all. A sensor on a constrained radio speaks MQTT for the same reason, and robots and self-driving stacks on ROS 2 move their world over DDS, the binary real-time pub/sub bus, because at those rates text would never keep up. The systems I work on in finance and anti-fraud run on the same instinct: Nasdaq’s market data ships as TotalView-ITCH, a sequenced stream of fixed-layout binary messages, an order book maintained by shipping the delta and letting every receiver rebuild the state (the plain-language version is the gentlest way in). It even flips agar.io’s choice, running big-endian with fixed-point integer prices, and the point survives the flip: when many parties have to agree on fast-moving numbers, the wire format is the performance.

10 A few things I keep learning about real-time

• Smallest type that still tells the truth. A position is two int16s, not two float64s — the shrink agar.io made across its own versions. Twelve bytes saved per record, a hundred records a frame, twenty-five frames a second.
• The order is the schema. Fixed widths at fixed offsets mean the reader never searches, never parses, never allocates: ~1.4 KB where JSON wanted ~4 KB. And one bit in a flags byte replaces 13 bytes of nick on every record of every snapshot after the first, because the cheapest byte is the one you already sent.
• Hold the world, stream the sector. Packing shrinks a message; culling decides there’s a small message at all, which is why the packet scales with a screen and not with the player count.
• Compress only what’s hot, and only with something that unpacks fast. LZ4 wraps opcodes 16 and 64 and nothing else, because that cost lands inside every frame the client draws.

I still haven’t rebuilt the game I lost, and reading somebody else’s wire isn’t the same as owning one. But what looks like a simple game about eating circles is nothing of the sort. Somewhere behind it are days of someone weighing a field width against a frame budget, deciding what you are allowed to see and how little it can cost to tell you, so that the whole thing arrives feeling instant and you never once have to think about it. That restraint is the craft. The game is very good; the engineering underneath it is better, and it is the same lesson my day job keeps making. Agree on the order ahead of time, ship only the delta, let each side rebuild the state from what it already has. Hold the entire world so nobody else has to, say only what the person on the other end can actually see, and say it in the fewest bytes that still tell the truth. The first .io game I built to learn that is ash. The tab that taught it to me again is still, as far as devtools is concerned, almost completely empty.