-
Hi,
I really enjoy reading and doing one 2d pose estimation project using PoolFormer as backbone, also love the idea of metaformer. Have you thought about pretraining the model using MAE? Would you…
-
**Your question**
Hello, as far as I know about Megatron, I've only seen padding mask for bert implementation.
Yet in Huggingface transformers library, the llama model should also take in the paddin…
-
## Description
After https://github.com/dmlc/gluon-nlp/pull/1356 (Thanks @szha and @leezu!), GluonNLP has now fully embraced the new Gluon 2.0 API. We will no longer need to worry about the `hybrid_f…
-
Hi,
I am wondering if at any point the training code for SEED-LLaMA will be made available?
-
Is there a guide for how to fine-tune DINO on a custom imagenet-formatted dataset? (after pretraining on custom data)
-
**Is your feature request related to a problem? Please describe.**
PR https://github.com/Project-MONAI/MONAI/pull/2253 implements a generic version of resnet for spatial 1/2/3D inputs. It'd be very u…
wyli updated
4 months ago
-
hi,it seems like that the dataset of pretrain stage1 and stage2 mentioned in the blip2 paper contains coco,cc3m,cc12m,sbu and laion ,but the config file only include coco and vg dataset.which is true …
-
hello. The command to run in the Filtering Data For Contrastive Pretraining section of https://github.com/nomic-ai/contrastors/tree/main/scripts/text is
```sh
torchrun --nproc-per-node= --dataset…
-
Hi guys,
I trained from scratch a new sentencepiece model on my pretraining dataset, however I still get unk tokens. Do you know why? I remember the last summer was working smoothly!
Specifically:…
ghost updated
3 years ago
-
Hi! Thanks for providing such a wonderful work.
I wonder have you tried a ResNet backbone without ImageNet pretraining?
Is it possible that a pre-trained model might become one of the keys of the p…