-
Hello!
I am trying to perform continued pretraining on the mbart.cc.25 pretrained checkpoint using the multilingual denoising objective. However, I am not sure how to prepare and pre-process the da…
-
Hello. Thank you for your amazing work.
I got a problem when I try to fine tune a pretrained model on my personal dataset(Dataset020). Following the steps in documentation/pretraining_and_finetuning.…
yy042 updated
1 month ago
-
I know the general recommendation is to leave the backbone frozen and train task-specific heads. However I'm interested in continuing pre-training to better fit the backbone features to my dataset. Is…
-
Hi, thx for your work! Do you plan to release the pretraining code? Like training dataset.
-
Running the pretraining example from GitHub fails when run in Google Colab.
```
!pip install 'litgpt[all]'
!mkdir -p custom_texts
!curl https://www.gutenberg.org/cache/epub/24440/pg24440.txt -…
-
Hi, does the SwinTransformer v2 do SimMiM pretraining? This is shown in the paper:
https://arxiv.org/pdf/2111.09883.pdf
If not, any plans to add/how difficult would it be to port?
-
Amazing model. Could you provide the codes to preprocess custom data and use it to train a model? I cannot use the provided scripts for running a model even with the indicated datasets and weights. Fo…
-
The link to pretraining weights on Mendeley data requires Elsevier login. But still not able to open the link after register and login to the Elsevier account
-
Thank you for open-sourcing the code! I didn't find descriptions about pretraining datasets in the paper. Was Starmie pertained on benchmark datasets?
-
Hello,
Thank you for your very interesting model.
I intend to use SparseBEV on a new dataset with only 1 frame , as a baseline (no previous temporal frame).
In the paper, on ablation study, y…