Training Parameters

[exitudio]

Thank you for open-sourcing your amazing work.

I cannot replicate the result of the first stage training (rvq). As you updated the configurations, I tried both but did not get FID as low as your pretrain model.

1. Batch size 256 and 50 epochs => FID 0.05146

   python train_vq.py --name rvq_name --gpu_id 1 --dataset_name t2m --batch_size 256 --num_quantizers 6  --max_epoch 50 --quantize_drop_prob 0.2

2. Batch size 512 and 500 epochs => FID 0.03358

   python train_vq.py --name rvq_name --gpu_id 1 --dataset_name t2m --batch_size 512 --num_quantizers 6  --max_epoch 500 --quantize_drop_prob 0.2

   ---

   I ran evaluation on your pretrain model FID is 0.019. Could it be due to high variance during training, or is there something specific I should try? Any suggestions would be greatly appreciated.

[EricGuo5513]

Hi, thanks for your interest. Could you use the configuration indicated in the opt.txt of the given checkpoint, including the learning scheduler? Please note that after all we will use the checkpoint with the best FID on the validation set.

On Fri, 5 Jan 2024 at 08:44, EXIT @.***> wrote:

Thank you for open-sourcing your amazing work.

I cannot replicate the result of the first stage training (rvq). As you updated the configurations, I tried both but did not get FID as low as your pretrain model.

1. Batch size 256 and 50 epochs => FID 0.05146

python train_vq.py --name rvq_name --gpu_id 1 --dataset_name t2m --batch_size 256 --num_quantizers 6 --max_epoch 50 --quantize_drop_prob 0.2

1. Batch size 512 and 500 epochs => FID 0.03358

python train_vq.py --name rvq_name --gpu_id 1 --dataset_name t2m --batch_size 512 --num_quantizers 6 --max_epoch 500 --quantize_drop_prob 0.2

---

I ran evaluation on your pretrain model FID is 0.019. Could it be due to high variance during training, or is there something specific I should try? Any suggestions would be greatly appreciated.

— Reply to this email directly, view it on GitHub https://github.com/EricGuo5513/momask-codes/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRYNB67AKKUJOFKQFR2OTLYNANXVAVCNFSM6AAAAABBOTMWQWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DONJWHE3TOMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[exitudio]

Thank you for your suggestion.

[weihaosky]

Thank you for your suggestion.

Hi, I have the same issue. Have you found out how to change the training parameters to replicate the performance of the pre-trained model?