Taming the Bot: Supervised Instruction Fine-Tuning Llama 3
A deep, hands-on walkthrough of supervised fine-tuning Llama 3.1 8B for customer support and a full analysis of what the model actually learned.
Deep-Learning
Anatomy of Trillion-Parameter Switchboards: Understanding Feedforward Blocks
Exploring the hidden layers of trillion-parameter switchboards: Feedforward Neural Networks and Activation Functions.
The Geometry of Meaning: Sine, ALiBi, RoPE, and HoPE
From Sinusoidal to RoPE and HoPE: How Transformers learn to process word order and sequence length.