search

cindyxinyiwang / deep-latent-sequence-model

Pytorch implementation of "A Probabilistic Formulation of Unsupervised Text Style Transfer" by He. et. al. at ICLR 2020
163 stars 26 forks source link

reduce eval time on bleu score? #10

Closed Timothy199707 closed 3 years ago

Timothy199707 commented 3 years ago

in line 405 of src/main.py:

dev_batch_size = hparams.valid_batch_size if hparams.beam_size == 1 else 1

causes the evaluation part done with a batch size of 1 everytime beam > 1, and causes the calculation time to be excessively long. Is there a reason for the beam size needed to be 1 (e.g, gpu memory I assume), or can I just change it to dev_batch_size = hparams.valid_batch_size instead?

Timothy199707 commented 3 years ago

I tried to run yelp and it take hours to finish bleu eval. I did some changes in the data_util file, but I'm not sure if it's related

jxhe commented 3 years ago

Line 405 of src/main.py was done deliberately since this repo doesn't have an efficient batched version of beam search when beam > 1 -- if you do beam > 1, then the model just translates the batch one by one and is very slow as you observed. In our experiments we always used beam=1

Timothy199707 commented 3 years ago

Oh, I see, thanks for replying