Closed Oakford closed 7 years ago
I am getting the same issue. I trained using
python train_multi_gpus.py
My hyperparmaters.py was modified slightly for 2 x GPUs and more epochs:
diff --git a/hyperparams.py b/hyperparams.py
index 5aaccab..8ac9032 100644
--- a/hyperparams.py
+++ b/hyperparams.py
@@ -35,13 +35,13 @@ class Hyperparams:
# training scheme
lr = 0.001 # Paper => Exponential decay
logdir = "logdir"
- batch_size = 32
- num_epochs = 200 # Paper => 2M global steps!
+ batch_size = 64
+ num_epochs = 2000 # Paper => 2M global steps!
loss_type = "l1" # Or you can test "l2"
num_samples = 32
# etc
- num_gpus = 1 # If you have multiple gpus, adjust this option, and increase the batch size
+ num_gpus = 2 # If you have multiple gpus, adjust this option, and increase the batch size
# and run `train_multiple_gpus.py` instead of `train.py`.
target_zeros_masking = False # If True, we mask zero padding on the target,
# so exclude them from the loss calculation.
Sorry for this. I forgot to change self.decoder_inputs to self.y. Pull now.
Fixed for me. Thanks!
Using the latest version, the program "eval.py" throws an error