-
您在文中并没有提到在imagenet上训练跑了多少epoch以及用了几块GPU,batch size是多大,还有优化器和学习率等参数。是否方便公布一下这些参数或者将预训练的log放出来?这会极大的帮助我们复现您的工作,非常感谢!
-
The paper mentions using Taskmaster as another dataset for pretraining. Is there code for preprocessing Taskmaster for training as well? I think there's only code for processing SGD.
-
Hello,
I loved reading your paper. However, I couldn't find a Supervised Metric Pretraining. Can I contribute by adding it?
Thank you
-
I forgot to tune the `num_workers` so the `train_bert()` ran for an extended time. I simply tuned the `num_worker` to `0` and completed training in less than 1 minute on a single RTX 3090 GPU.
-
"Please ensure that the architectures match.".format(filename)
Exception: Cannot load model parameters from checkpoint /content/self-supervised-speech-recognition/wav2vec_small_960h.pt; please ensure…
-
- Convolutional Autoencoders (http://blog.keras.io/building-autoencoders-in-keras.html)
- Convolutional real values RBMs
Look in to implementing one or the other . Autoencoders in keras seem straight…
-
### Describe the bug
Hello, I am pretraining Wav2vec2 following the instructions on this [page](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriSpeech/self-supervised-learning/wa…
-
### Description
pretraining
### (Optional:) Please add any files, screenshots, or other information here.
_No response_
### (Required) What is this issue most closely related to? Select one.
Choo…
-
Investigating Continual Pretraining in Large Language Models: Insights and Implications
Examining Forgetting in Continual Pre-training of Aligned Large Language Models
Towards Incremental Learni…
rgtjf updated
1 month ago
-
It appears that config/pretrain-alldata-base.json is not your paper pretraining configuration. There is no cls_concat setting in this configuration file, so it uses the default value. As a result, unl…