mnist result is not good

[jhyoocoder]

I got the performance of L2L better than that of Adam about quadratic problem.

1. python train.py --problem=quadratic --save_path=./quad

   Run on GPU again
   Epoch 10000
   Log Mean Final Error: -1.08
   Mean epoch time: 0.16 s
   EVALUATION
   Log Mean Final Error: -1.08
   Mean epoch time: 0.05 s
   Removing previously saved meta-optimizer
   Saving meta-optimizer to ./quad

2. Evaluate with L2L; python evaluate.py --problem=quadratic --optimizer=L2L --path=./quad

   Epoch 100
   Log Mean Final Error: -0.70 # works better than Adam optimizer below
   Mean epoch time: 0.17 s

3. Evaluate with Adam; python evaluate.py --problem=quadratic --optimizer=Adam --path=./quad

   
   Epoch 100
   Log Mean Final Error: -0.06
   Mean epoch time: 0.09 s

However, the mnist problem seems not to reproduce the same result with that of the paper.
4. python train.py --problem=mnist --save_path=./mnist

Epoch 10000 Log Mean Final Error: -0.47 Mean epoch time: 0.76 s EVALUATION Log Mean Final Error: -0.42 Mean epoch time: 0.31 s

5. python evaluate.py --problem=mnist --optimizer=L2L --path=./mnist

Epoch 100 Log Mean Final Error: 0.18 Mean epoch time: 0.60 s

6. python evaluate.py --problem=mnist --optimizer=Adam --learning_rate=[0.1| 0.3 | 0.01 | 0.003 | 0.001] 

0.1 | 0.3 | 0.01 | 0.003 | 0.001
-- | -- | -- | -- | --
Epoch 100 Log Mean Final Error: -0.31 Mean epoch time: 0.23 s | Epoch 100 Log Mean Final Error: 0.24 Mean epoch time: 0.23 s | Epoch 100 Log Mean Final Error: -0.31 Mean epoch time: 0.23 s | Epoch 100 Log Mean Final Error: 0.08 Mean epoch time: 0.24 s | Epoch 100 Log Mean Final Error: 0.29 Mean epoch time: 0.23 s

What did i do wrong? Could you help me? I resolved “TypeError: The two structures don't have the same nested structure.” with the solution mentioned in the following issue:
https://github.com/deepmind/learning-to-learn/issues/22

Thank you in advance.

[jhyoocoder]

Resolved by myself. It was due to tensorflow version mismatch. I've worked on tf 1.14, but I succeeded after downgrading to tf1.5. Thanks!

[ko3n1g]

@jhyoocoder Can elaborate on your requirements.txt? I am facing the same issue - no reproducibility on mnist evaluate, am getting the same results like you initially

[ko3n1g]

solved with #34