LiveLLM · Notes

Next-token notes

My notes on how an LLM picks the next word — tokens, embeddings, attention, softmax — one step at a time.

Eight steps, from a prompt to one new word. It plays by itself, or step through it at your own pace.

How it works

1These are notes I made while learning it, not a course.
2Every number in it is simulated to show the idea; none come from a real model.

More demos

All demos

LiveDatabases · Built from scratch

QueryLite: a SQL database

A SQL database written from scratch — parser, B+ tree indexes, a query planner, joins and transactions — running live on 22,000 rows in your browser.

Try it

LiveDistributed systems · Built from scratch

Raft consensus, live

The algorithm that keeps etcd and Kubernetes consistent. Five servers elect a leader and replicate a log — crash them and split the network while it runs.

Try it

LiveReal-time · Built from scratch

Live collaborative editor

Real-time editing with no server. Three devices share a note — take one offline, edit everywhere, reconnect, and they merge. Open a second tab and it syncs live.

Try it