LLM CLASSIC PAPERS ROADMAP

大模型经典论文路线图

Self-Attention
Encoder-Decoder
Parallelization

Transformer

Human Preferences
Reward Learning
Reinforcement Learning

RLHF

Generative Pre-training
Decoder-only
Fine-tuning

GPT-1

Masked LM
Bidirectional
Pre-training

BERT

Zero-shot
WebText
Scaling

GPT-2

Text-to-Text
Transfer Learning
Unified Framework

T5

175B Parameters
Few-shot Learning
In-context Learning

GPT-3

Code Generation
GitHub Copilot
HumanEval

CodeX

Competitive Programming
Code Reasoning
Codeforces

AlphaCode

Compute-Optimal Scaling
Data-Centric Training

LLaMA-1

RLHF
GQA
长上下文
高效推理

LLaMA-2

GQA
NTK-aware
LogN-Scaling

Qwen-1