-
in case people would like to contribute adding another english accent or another languages, which documents should they refer ?
-
Hello, would it be possible to also release the pretraining dataset ( used for TSmixup), and maybe a mention of a successful training recipe.
I would like to try to pretrain from scratch as well, …
-
Hi! I can't find the "scripts" folder, hence I can't find "run_pretraining.sh" file. How should I pretrain the model?
-
Thank you for sharing a good paper.
In the paper, if you look at the result of the experiment in Fig 6,
there is a vision only test result,
and it was said that the performance would be improved…
-
In the Masked Pretraining section, there seems to be an issue with the way the CLIP model is loaded. In the extract.ipynb notebook, the code model, _ = clip.load("ViT-B/16", device='cpu') is used, but…
-
### feature
Hi, I am trying to redo the pretrain step as you described in the readme doc. The training loss converges pretty fast. I find the logs in wandb and it turned out to be only containing the…
-
### Description
# Arguments
T5_SIZE='xl' #$1 # Model size (small, base, large)
PREC="bfloat16" #"$2" # Precision (float32, float16, bfloat16)
NUM_GPUS=8 #$3 # Number of GPUs (1, …
-
In Table 3 & 4, is the same dataset used during pre-training and fine-tuning? Or does the fine-tuning only happened on ImageNet-1k dataset?
-
Hi,
For the paper https://arxiv.org/pdf/2310.01218.pdf , the following is mentioned in pretraining section :
```
For efficiency, we first train SEED-LLaMA using LoRA [32] tuning and together o…
-
I'm interested in pretraining DINOv2 on my own custom dataset, starting from the ImageNet-1K pretrained weights. Specifically:
1. Is it possible/recommended to continue pretraining DINOv2 on a cust…