THUNLP-MT / THUMT

An open-source neural machine translation toolkit developed by Tsinghua Natural Language Processing Group
BSD 3-Clause "New" or "Revised" License
701 stars 197 forks source link

Do you have an instruction manual for the pytorch version? #82

Open Felixgithub2017 opened 4 years ago

Felixgithub2017 commented 4 years ago

Some commands for additional parameters are not working in the pytorch version, so, do you have an pytorch-oriented manual?

For example:

--parameters=batch_size=15000,device_list=[0,1],update_cycle=2,train_steps=2000000,keep_checkpoint_max=5,shared_embedding_and_softmax_weights=True,shared_source_target_embedding=True

raise ValueError("Could not parse hparam %s in %s" % (name, values)) ValueError: Could not parse hparam shared_embedding_and_softmax_weights in batch_size=15000,device_list=[0,1],update_cycle=2,train_steps=2000000,keep_checkpoint_max=5,shared_embedding_and_softmax_weights=True,shared_source_target_embedding=True

Felixgithub2017 commented 4 years ago

image The initial loss is inf, and turned to normal after around 200 steps

Playinf commented 4 years ago

In the above example, you should set shared_embedding_and_softmax=true instead of shared_embedding_and_softmax=True. The document of PyTorch implementation will be uploaded soon. We have tested our implementation on several datasets, but we do not observe inf loss problem.