How a GPU actually executes the maths.

A GPU can do arithmetic far faster than it can fetch the numbers to do it on. An A100's FP32 units need 12.5 flops of work on every byte from its HBM just to keep busy. Every fast kernel is a way of closing that gap: reuse data close to the ALUs, fetch it in whole transactions, avoid serialising on shared memory, keep enough warps in flight.

Each chapter here is built around an animation, and every frame is computed by a small GPU execution model that is tested against a Python reference. The hardware figures come from NVIDIA's whitepapers and datasheets, the CUDA Programming Guide and published microbenchmarks.