Research Note

Disclaimer: The notes are primarily intended for my personal learning and may not be entirely accurate.

• Loss spikes in pretraining

• Information Exponents and Neural Networks
  This note is made for the reading group of multi-index model[Section 5.1], including DLS22 and BES+22.

• Physics of LLM
  Based on Zeyuan Allen-Zhu’s Physics of LLM work.

• Algorithmic Aspects of Machine Learning
  Based on Ankur Moitra’s Algorithmic Aspects of Machine Learning.

• PhD Statement of Purpose

• Some Random Notes