The Geometry of Meaning: Sine, ALiBi, RoPE, and HoPE
From Sinusoidal to RoPE and HoPE: How Transformers learn to process word order and sequence length.
Tech
My First Post
A brief introduction to Vectors & Verbs and formatting verification.
From Sinusoidal to RoPE and HoPE: How Transformers learn to process word order and sequence length.
A brief introduction to Vectors & Verbs and formatting verification.