bbycroft / llm-viz

3D Visualization of an GPT-style LLM
https://bbycroft.net
3.26k stars 366 forks source link

Brendan Bycroft's Home Page & Projects

This repository contains my (Brendan's) homepage, as well as a number of non-trivial projects.

They are kept in a single repository for ease of deployment, as well as sharing a bunch of js utils which are otherwise a pain to share around.

Projects

The main projects are:

LLM Visualization

This project displays a 3D model of a working implementation of a GPT-style network. That is, the network topology that's used in OpenAI's GPT-2, GPT-3, (and maybe GPT-4).

The first network displayed with working weights is a tiny such network, which sorts a small list of the letters A, B, and C. This is the demo example model from Andrej Karpathy's minGPT implementation.

The renderer also supports visualizing arbitrary sized networks, and works with the smaller gpt2 size, although the weights aren't downloaded (it's 100's of MBs).

CPU Simulation (WIP; not exposed yet!)

This project runs 2D schematic digital circuits, with a fully fledged editor. The intent is to add a number of walkthroughs, showing things such as:

Running Locally

  1. Install dependencies: yarn
  2. Start the dev server: yarn dev