Open anirudha16101 opened 2 years ago
I am tr.ying to train the model for custom data which contains the data in format: folder/img1.png folder/img1.txt
python3 train.py --folder FeatureStore --batch_size 512 --gpus 1
I hard code the warmup steps = 100 and first cycle steps = 500 (usually set value showing in the user guide), then it was fixed.
I actually want to know what information is in this.txt file, is it category information? caption or QA?
Hi, Can somebody please help me out here why this error is coming?
Using native 16bit precision. GPU available: True, used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs /home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/trainer/configuration_validator.py:101: UserWarning: you defined a validation_step but have no val_dataloader. Skipping val loop rank_zero_warn(f"you defined a {step_name} but have no {loader_name}. Skipping {stage} loop") Path FeatureStore LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] Traceback (most recent call last): File "train_finetune.py", line 33, in
main(args)
File "train_finetune.py", line 23, in main
trainer.fit(model, dm)
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 552, in fit
self._run(model)
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 873, in _run
self.accelerator.setup(self, model) # note: this sets up self.lightning_module
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu.py", line 42, in setup
return super().setup(trainer, model)
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/accelerators/accelerator.py", line 88, in setup
self.setup_optimizers(trainer)
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/accelerators/accelerator.py", line 331, in setup_optimizers
trainer=trainer, model=self.lightning_module
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 223, in init_optimizers
return trainer.init_optimizers(model)
File "/home/ubuntu/.local/lib/python3.6/site-packages/pytorch_lightning/trainer/optimizers.py", line 34, in init_optimizers
optim_conf = model.configure_optimizers()
File "/home/ubuntu/clip/train-CLIP/models/wrapper.py", line 343, in configure_optimizers
warmup_steps=2000
File "/home/ubuntu/.local/lib/python3.6/site-packages/cosine_annealing_warmup/scheduler.py", line 27, in init
assert warmup_steps < first_cycle_steps
AssertionError