Native
Sign in
For people who read papers
Read papers.
Make them
stick.
Native turns any paper into a calm study session — read, ask, quiz, code, and converse.
Get started — it’s free
3.2.1 · Scaled Dot-Product Attention
The input consists of
queries and keys
of dimension d
k
, and values of dimension d
v
.
Quick check · question 2/5
Why scale dot products by 1 ⁄ √d
k
?
Stops softmax saturation
Speeds up inference
Avoids vanishing gradient