-
Hi,
I tried to further pretrain XLNet in domain specific corpus like what is recommanded in BERT. But I got worse result. Has anyone tried further pretraining? Does it work?
Thanks!
-
### Describe the feature
I am following [Colossal-LLaMA-2](https://github.com/hpcaitech/ColossalAI/tree/main/applications/Colossal-LLaMA-2) to continue pretraining. I am using a 8x a100 80G node. And…
-
Hi, could you please share some caption examples for pretraining on Audioset? I'm a little confused about the [mask] token setting for clip text encoder.
-
Is there a guide for how to fine-tune DINO on a custom imagenet-formatted dataset? (after pretraining on custom data)
-
Is there a training code for the text-2-image model for stable diffusion (all training steps including pretraining of Autoencoder with KL)? I could only find the inference command in the repo.
Than…
-
When I install the environments, a lot of packages conflict each other.
could you please release a docker for training?
Recently, I have read your new work “Self-Supervised Pretraining for Large-S…
-
## Problem statement
1. Unlabeled data를 간접적으로 활용하는 방법 외에 NLP에서 활용하는 것과 같이 더 직접적으로 활용하는 task-agnostic한 방법을 찾는다.
- NLP에서의 활용: unsupervised pretrain -> supervised fine-tune
2. unlabeled data를 활용해…
-
## 🐛 Bug
When I run the fairseq-hydra-train script with pretraining config (large), but loading the XLSR 53 checkpoint, I get a KeyError: 'max_exp_avg_sq' on fairseq/optim/adam.py after some traini…
-
Dear Sir/Madam,
Thank you for such a great repo, but I am confused by the result for EC downstream task result. The result of multiview contrast in paper "PROTEIN REPRESENTATION LEARNING BY GEOMETR…
-
Hi, what is your mvm accuracy of pretrained model? I only got about 30% when pretraining and wanted to know if that is normal?