LLMを理解する必読論文12選

1. Attention Is All You Need The paper that introduced the Transformer. We will learn how self-attention replaces recurrence, and why this one architecture powers every modern LLM. I decoded this paper, you can read these articles: Amit Shekhar @amitiitbhu · Apr 7 Article Decoding Transformer Architecture In this blog, we will learn about the Transformer architecture by decoding it piece by piece - understanding what each component does, how they work together, and why this architecture powers every... 5 115 615 73K Amit Shekhar @amitiitbhu · Apr 3 Article Math behind Attention - Q, K, and V In this blog, we will learn about the math behind Attention: Query(Q), Key(K), and Value(V) with a step-by-step numeric example. We will cover the following: The Attention Formula Setting Up: From... 9 224 1.3K 192K Amit Shekhar @amitiitbhu · Apr 5 Article Math behind √dₖ Scaling Factor in Attention In this blog, we will learn about why we scale the dot product attention by √dₖ in the Transformer architecture with a step-by-step numeric example. If you have already read our blog on Math behind... 3 56 347 71K Amit Shekhar @amitiitbhu · Apr 13 Article Feed-Forward Networks in LLMs In this blog, we will learn about Feed-Forward Networks in LLMs - understanding what they are, how they work inside the Transformer architecture, why every Transformer layer needs one, and what role... 1 17 85 18K 2. BERT We will learn masked language modeling and why bidirectional context made BERT the default for understanding and classification tasks. 3. GPT-3: Language Models are Few-Shot Learners We will learn how a 175B-parameter decoder-only model unlocks in-context learning, picking up new tasks from just a few examples in the prompt. 4. Scaling Laws for Neural Language Models We will learn how loss decreases predictably with compute, data, and parameters, so we can plan model size and training before burning a single GPU hour. 5. Chinchilla We will learn that most large models were undertrained, and that roughly 20 tokens per parameter is compute-optimal. At a fixed compute budget, a smaller model trained on more tokens beats a bigger one trained on less. 6. InstructGPT The paper behind the methods that power ChatGPT. We will learn how RLHF turns a raw text predictor into a helpful assistant through supervised fine-tuning, reward modeling, and PPO. 7. Chain-of-Thought Prompting We will learn how asking a model to think step by step dramatically improves reasoning on math, logic, and multi-step problems. 8. Retrieval-Augmented Generation We will learn how combining a retriever with a generator lets models answer using fresh, factual documents, without retraining. 9. LoRA: Low-Rank Adaptation We will learn how LoRA cuts trainable parameters by 10,000x by training tiny rank-decomposition matrices instead of the full weights. This is the foundation that QLoRA later used to fine-tune 70B models on a single GPU. 10. LLaMA We will learn how a well-trained 13B model can outperform GPT-3 on most benchmarks, and how open weights reshaped the entire research landscape. 11. FlashAttention We will learn how IO-aware attention cuts memory usage and speeds up training, without changing the math. I decoded this paper, you can read the article: Amit Shekhar @amitiitbhu · Apr 11 Article Decoding Flash Attention in LLMs In this blog, we will learn about Flash Attention by decoding it piece by piece - understanding why standard attention is slow, what makes Flash Attention fast, how it uses GPU memory cleverly, and... 21 109 19K 12. DPO: Direct Preference Optimization We will learn how to align a model on preference data directly, without a reward model and without reinforcement learning.