Attention Is All You Need by Vaswani et al., 2017 (NeurIPS)
BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding by Devlin et al., 2019 (NAACL)
GPT2: Language Models are Unsupervised Multitask Learners by Radford et al., 2019
T5: Exploring the Limits of Transfer Learning with a Unified TexttoText Transformer by Raffel et al., 2020 (ICLR)
3、阅读和理解论文
Attention Is All You Need:介绍了Transformer模型,强调了自注意力机制的重要性。
BERT:提出了双向编码器表示从Transformer(BERT),通过预训练和微调在多个NLP任务上取得了很好的效果。
GPT2:展示了语言模型的强大能力,特别是在无监督学习方面的表现。
T5:统一了文本到文本的任务,并在多种任务上表现出色。