-
I want to pre-train BERT on Urdu. There are a few things which I would like to know.
BERT multilingual model does not contain all the tokens(tokenized words) in vocab.txt. So if I want to pre-train,…
-
## What I did?
* Checked out the `pretrain-lm` branch because it has clear instructions how to pretrain LM (#57).
* Installed required packages.
* Executed `bash prepare_wiki.sh de`
* Executed …
-
Thank you for opening and maintaining this project.
I want to reference your paper and reproduce your experimental results. However, I find that the region-level files for pretraining (eg. 201511_…
-
Hey,
I want to pretrain and benchmark small and base versions of Electra for the Arabic and Persian languages. As mentioned in the run_pretraining python file, only "base" and "large" model_size ar…
-
Hello, I'd like to conduct pre-training on the st-mem model using the same dataset. I have all the datasets ready. My question pertains to the contents of st_mem/configs/pretraining.yaml. Could you pl…
-
For the following two checkpoints listed in [optimus_finetune_language_models.md](https://github.com/ChunyuanLI/Optimus/blob/master/doc/optimus_finetune_language_models.md):
> beta=0, latent size =…
-
Hi Yinyu:
I tried to pretrain MGNet with `python main.py configs/mgnet.yaml --mode train` and test it with `python main.py configs/mgnet.yaml --mode test`.
However, after 50 epochs of training,…
-
Hello!
Thank you so much for developing and releasing this model to the public. As a native Arabic speaker, I highly appreciate your efforts in enriching our beautiful language.
I have the follo…
-
Thanks for the great work!
Can you release the CAE-S checkpoint model?
Thank you!
-
### Question
using script scripts/v1_5/pretrain.sh, I conducted llava pretraining using some task-specific image-text pair, now I am thinking of testing the pretrained model, and wondering how can …