-
can't find task.json at task_group_json="./data/en/pretraining/roberta_skep_large_en/task.json"
-
Hi,
When running wiki103 gpt2-m and gpt2-l baseline pretraining experiments,
`python run.py experiment=wt103/gpt2m`
and
`python run.py experiment=wt103/gpt2l`
will receive non-converge error.
…
-
Thank you for your great work!
I'm trying to train a model for ImageNet1K classification task.
In my computational environment, one epoch requires 600,540 training steps and the training advances …
-
Can we use our private data to train this foundation model ?
-
The mask ratio for EEG pretraining in your paper is 75%, but in your code (config.py), the ratio is 10%. Could you please tell me the mask ratio you used for EEG pretrain? Many thanks.
HLSUD updated
9 months ago
-
Hey! great work and congratulations on NAACL acceptance!
Can you specify how do you determine the number of intermediate pretraining steps for TAPT/DAPT/SDPT?
Thanks in advance.
-
## Description
Add the CC-100 corpus that can be used for pretraining to `nlp_data`.
http://data.statmt.org/cc-100/
-
## Feature request
After semi-supervised pretraining, can we do light-weighted fine-tunning or few-shot learning instead of classification?
**What is the expected behavior?**
Instead of fine-…
-
Could you please check the implementation steps you provided in the README file?
I followed your instructions but find it very hard to reproduce this work, someerrors would come out like version in…
-
paper shows that they pretraining use all data in a unsupervised way.I'm wondering where you pretrain your net in your codes.