[Modeling] Release a 615M HBS (Croatian, Bosnian, Serbian) Open-NLLB checkpoint - Githubissues

gordicaleksa / Open-NLLB

Effort to open-source NLLB checkpoints.

MIT License

419 stars 37 forks source link

[Modeling] Release a 615M HBS (Croatian, Bosnian, Serbian) Open-NLLB checkpoint #15

Open gordicaleksa opened 1 year ago

gordicaleksa commented 1 year ago

Our goal is to release an open-source, 3.3B, dense checkpoint that does machine translation for 202 languages from the NLLB project.

To get to that point let's first start by releasing OSS checkpoints with smaller scale and smaller language scope.

The goal here will be to release a model with following properties:

(small scale) 615 M parameters
(smaller language scope) Support only Croatian, Bosnian, Serbian, Montenegrin languages + English.