search

HazyResearch / m2

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
Apache License 2.0
517 stars 42 forks source link

Multilingual? #20

Open RibinMTC opened 5 months ago

RibinMTC commented 5 months ago

Are you planning to release a multilingual version of this model? Could I finetune the current m2_bert model on german data?

DanFu09 commented 4 months ago

We currently are not planning on training a multilingual version, but let me know if the finetuning works! It's currently using the BERT tokenizer, so I'm not sure how well that maps to German.