search

acoladgroup / OpenNMT-py

Open Source Neural Machine Translation in PyTorch
http://opennmt.net/
MIT License
8 stars 6 forks source link

fix setup.py and handle special case with zero gpus. #2

Closed se4u closed 6 years ago

se4u commented 6 years ago

Thanks for the effort to make onmt pth4 compatible.

It may be good to test the successful completion of the MT tutorial which is run nightly, or weekly, but not on every commit.

se4u commented 6 years ago

This comment is unrelated to the PR but rather related to the multigpu training script. I noticed that the performance of the multigpu training is quite a bit lower than the single gpu training after the same number of epochs. I used the standard data that is bundled with the OpenNMT package. Is this expected ?

pth4.ubiquous.multigpu.log pth4.ubiquous.gpu.log

vince62s commented 6 years ago

yes it is expected. the batch size is 4 times bigger. in theory, you could expect 1 GPU batch size 4096 to be comparable with 4GPU batch size 1024 (in token mode) you can do the same in sentence mode. (ie 16 or 4x16)

se4u commented 6 years ago

ah I forgot about the batch size. thanks for clearing this up.

On Thu, 31 May 2018, 23:50 Vincent Nguyen, notifications@github.com wrote:

yes it is expected. the batch size is 4 times bigger. in theory, you could expect 1 GPU batch size 4096 to be comparable with 4GPU batch size 1024 (in token mode) you can do the same in sentence mode. (ie 16 or 4x16)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Ubiqus/OpenNMT-py/pull/2#issuecomment-393780020, or mute the thread https://github.com/notifications/unsubscribe-auth/ACMdtJhR8BDAzQWGsl_UYvlaNdP95zyiks5t4OQvgaJpZM4UVg-X .

vince62s commented 6 years ago

ok thanks for the PR, for some reason I edited the changes directly when people opened issues. but thanks again, sorry.