Notes

PCA

PCAの基礎と実装について

GPT-OSS Attnention Sink

The bias term for preventing attention sink in GPT-OSS may have other effects than just preventing attention sink.

RMSNorm and LayerNorm

Relationship between RMSNorm and LayerNorm and rewritten LayerNorm using centering matrix.