-
Right now, the code is fixed to input sizes of `224,224px` and patch sizes of `32px` (like configured in the paper). The code should be adapted to allow different sizes when pretraining.
-
As far as I know, one can pretrain fine using TinyStories:
```python
litgpt/pretrain.py --data litgpt.data.TinyStories
```
Should we add this to the documentation?
Right now, we only have …
rasbt updated
5 months ago
-
Hi,
We found that video text joint loss in pretraining is calculated from masked video and text. Why not use the origin video and text like retrieval finetune?
https://github.com/microsoft/UniVL/blo…
-
**Describe the bug**
The provided pre-trained Swin-UNETR weights do not load into a newly instantiated SSLHead model object. The naming scheme for the model state_dict keys is different between the p…
-
Hi, thank you for your work while I was training at 4090 Single card, raining isic18 with the pretraining model pre_trained_weights/vssmsmall_dp03_ckpt_epoch_238.pth could not get the miou result of 8…
-
Any one know where to get them?
Thank you and thank you.
-
Thank you for your nice work.
I try to read the pretraining doc and find that there are some errors. In PRETRAIN_GUIDE.md, you mention that the script for data conversion is in [scripts/data/megatr…
HYZ17 updated
1 month ago
-
Hi, thank you for this great work.
I tried the pretraining model on the **NTU60** following the setup instructions. The training **loss** is highly **oscillating**, would you please explain this be…
-
I am trying to reproduce your results on miniImageNet and tieredImageNet. I can reproduce the result of S2M2 with the given rotation weights on miniImageNet. But when I train the rotation task by myse…
-
Hi,
I have run the pretraining.py data on domain specific text.
I gave 9 lakh sentences text file, batch size 32, learning rate 2e-5, num of train steps 10000.
masked lm accuracy - 69%
Aft…