Technical Notes

Longer written notes for readers who want the detail: the maths, the code and the sources. Each one goes with a video, and none is required to follow it.

Level

Series

Newest first

Showing 03 of 03

TN-03[DD MMM YYYY]12 min read
The softmax function, worked by hand
Turning a list of raw scores into probabilities, one step at a time, with a calculator.
MathsGoes with D1 // Standard DeviationPlaceholder post
TN-02[DD MMM YYYY]9 min read
Inside a tokenizer: byte-pair encoding step by step
How a tokenizer learns its vocabulary by merging the most common pairs of characters.
IntermediateGoes with Ep2 // LLM 101Placeholder post
TN-01[DD MMM YYYY]6 min read
Why hallucinations aren’t bugs
A model that predicts likely text will sometimes produce likely text that is false.
IntroductoryGoes with Ep3 // LLM 101Placeholder post

Technical Notes

Newest first

The softmax function, worked by hand

Inside a tokenizer: byte-pair encoding step by step

Why hallucinations aren’t bugs