search

bigscience-workshop / multilingual-modeling

BLOOM+1: Adapting BLOOM model to support a new unseen language
https://arxiv.org/abs/2212.09535
Apache License 2.0
69 stars 15 forks source link

Adding Language Post-Training: MAD-X Adapters #6

Closed yongzx closed 2 years ago

yongzx commented 2 years ago

Extend GPT-2 to Korean and German by

  1. Train MAD-X Language Adapters with pre-finetuned embedding layers.
  2. Train MAD-X Language Adapters along with GPT-2 embedding layers.
yongzx commented 2 years ago

Added support for including language adapters for BigScience models (13 language version).