search

Gitsamshi / WeakVRD-Captioning

Implementation of paper "Improving Image Captioning with Better Use of Caption"
32 stars 7 forks source link

Confirming Training Setting Information #3

Closed tjuwyh closed 4 years ago

tjuwyh commented 4 years ago

Hi, Gitsamshi! Thanks for your work and the kind reply in previous issus. I just wanted to confirm some simple information about training setting, as I didn't see them in the paper/repo.

  1. I am running on a single 1080Ti and the batch size is set to 64, it consumes around 10GB of GPU memory and that each epoch takes ~1 hour in the XE step. Does this match your training time/memory usage?
  2. Besides, I didn't use the raw batch size (128) since would run out of the memory of GPU, would this change affect the performances a lot? Because I found that when I training a model under this setting, the performance reach a plateau (cider: 1.01) early (about 3~4 epochs). Does it means I did something wrong?
  3. By the way, would you be so generous to share some advice or tricks if I wish to reproduce the performances in the paper?
Gitsamshi commented 4 years ago

Thank you for asking. 1 Yes, I run it on Titan with 24 GB GPU, the time per epoch is within a reasonable range 2 No, it won't affect results much. And it would reach around 1.02 in 3~4, 1.10 in 10, 1.15 in 20, 1.16 in 30, 1.165 in 40 epochs on the validation set, Just keep training for more epochs (35-40) to get the results in the paper.

tjuwyh commented 4 years ago

Thank you for asking. 1 Yes, I run it on Titan with 24 GB GPU, the time per epoch is within a reasonable range 2 No, it won't affect results much. And it would reach around 1.02 in 3~4, 1.10 in 10, 1.15 in 20, 1.16 in 30, 1.165 in 40 epochs on the validation set, Just keep training for more epochs (35-40) to get the results in the paper.

Thanks for you kind reply, it really helps me out.