Notes

RoPE Implementation

October 3, 2025

How RoPE is implemented in practice.

メモ: PCA

August 6, 2025

PCAの基礎と実装

Folding weights in transformers

May 12, 2025

LogitLens without bias

July 24, 2024

Removing bias term from LogitLens.

RMSNorm and LayerNorm

July 12, 2024

Relationship between RMSNorm and LayerNorm and rewritten LayerNorm using centering matrix.