논문 추천리스트 (최신반)

[hyunwoongko]

2019년 이후 논문만 추천해주세요~ (2019년 논문부터 가능)

[KimDaeUng]

Few-shot Learning in NLP

• Trapit Bansal, Rishikesh Jha, and Andrew McCallum. 2019. Learning to few-shot learn across diverse natural language classification tasks. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5108–5123.
• Geng, R.; Li, B.; Li, Y.; Sun, J.; and Zhu, X. 2020. Dynamic Memory Induction Networks for Few-Shot Text Classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1087–1094.
• Trapit Bansal, Rishikesh Jha, Tsendsuren Munkhdalai, Andrew McCallum. 2020. Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 522-534.

[hyunwoongko]

@KimDaeUng 이 공간에는 2019년 이후 논문만 업로드 해주시길 바랍니다. 2018년에 발표된 논문들은 #2 로 옮겨두었습니다. :)

[KimDaeUng]

@hyunwoongko 주제만 보고 정리하다보니 실수로 18년도 것까지 올려버렸네요. 다음부턴 연도 구분해서 올리겠습니다. 감사합니다! :D

[Eunhui-Kim]

Language modeling with knowledge graph

Shin Agarwal, Heming Ge, Siamak Shakeri, Rami AI-Rfou, “Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language model pre-training Corpora,” NAACL 2021. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang, “REALM: Retrieval-Augmented Language Model Pre-training,” ICML 2020. Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng and Ping Wang, "K-BERT: Enabling Language Representation with Knowledge Graph," AAAI 2019.

[toriving]

Prompt engineering

Toriving

• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• WARP: Word-level Adversarial ReProgramming
• Prompt Programming for Large Language Models:Beyond the Few-Shot Paradigm
• Calibrate Before Use:Improving Few-Shot Performance of Language Models
• How Many Data Points is a Prompt Worth?
• GPT Understands, Too
• The Power of Scale for Parameter-Efficient Prompt Tuning
• Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

Hyunwoongko

• 동주 형, 여기에 논문 몇개 추가 좀 해놓을게 (같은 주제여서 여기 코멘트에 추가함)
• It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
• AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
• Parameter-Efficient Transfer Learning for NLP

[hyunwoongko]

GPU optimization & Large-scale LM

• Megatron LM
• Deep Speed
• ZeRO
• ZeRO-infinity
• ZeRO-offload
• Progressive Layer Dropping
• 1-bit Adam
• 1-bit LAMB
• GShard
• Switch Transformers

---

1 Bit LAMB 제외하고 전부 리뷰 완료.

[chloamme]

ByT5: Towards a token-free future with pre-trained byte-to-byte models True Few-Shot Learning with Language Models Why AI is Harder Than We Think FNet: Mixing Tokens with Fourier Transforms Fabula Entropy Indexing: Objective Measures of Story Coherence Parallel Corpus Filtering via Pre-trained Language Models