[Learned_optimizer] Error when running metarun.py

ghost commented 6 years ago

Please go to Stack Overflow for help and support:

http://stackoverflow.com/questions/tagged/tensorflow

Also, please understand that many of the models included in this repository are experimental and research-style code. If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

What is the top-level directory of the model you are using: ~/models/research/learned_optimizer
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v1.4.0-19-ga52c8d9 1.4.1
Bazel version (if compiling from source):
CUDA/cuDNN version:6.0
GPU model and memory: Titan xp
Exact command to reproduce: python metarun.py

Describe the problem

I was trying to run the training script 'metarun.py' but encountered errors shown as below

Source code / logs

Traceback (most recent call last): File "metarun.py", line 394, in tf.app.run() File "/home/rvl224/anaconda3/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "metarun.py", line 388, in main callbacks=[]) File "/home/rvl224/models/research/metaopt.py", line 184, in train_optimizer train_output = opt.train(problem, dataset) File "/home/rvl224/models/research/learned_optimizer/optimizer/trainable_optimizer.py", line 347, in train swap_memory=True, shape_invariants=invariants) File "/home/rvl224/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants) File "/home/rvl224/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop pred, body, original_loop_vars, loop_vars, shape_invariants) File "/home/rvl224/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2597, in _BuildLoop nest.assert_same_structure(list(packed_vars_for_body), list(body_result)) File "/home/rvl224/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/nest.py", line 222, in assert_same_structure % (len_nest1, nest1, len_nest2, nest2)) ValueError: The two structures don't have the same number of elements.

First structure (120 elements): [<tf.Tensor 'while/Identity:0' shape=() dtype=int32>, <tf.Tensor 'while/Identity_1:0' shape=() dtype=float32>, [<tf.Tensor 'while/Identity_2:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_3:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_4:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_5:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_6:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_7:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_8:0' shape=(12, 1) dtype=float32>], [<tf.Tensor 'while/Identity_9:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_10:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_11:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_12:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_13:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_14:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_15:0' shape=(12, 1) dtype=float32>], [[<tf.Tensor 'while/Identity_16:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_17:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_18:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_19:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_20:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_21:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_22:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_23:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_24:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_25:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_26:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_27:0' shape=(11, 10) dtype=float32>, <tf.Tensor 'while/Identity_28:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/Identity_29:0' shape=(11, 1) dtype=float32>], [<tf.Tensor 'while/Identity_30:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_31:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_32:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_33:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_34:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_35:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_36:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_37:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_38:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_39:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_40:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_41:0' shape=(3, 10) dtype=float32>, <tf.Tensor 'while/Identity_42:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/Identity_43:0' shape=(3, 1) dtype=float32>], [<tf.Tensor 'while/Identity_44:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_45:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_46:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_47:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_48:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_49:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_50:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_51:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_52:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_53:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_54:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_55:0' shape=(9, 10) dtype=float32>, <tf.Tensor 'while/Identity_56:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/Identity_57:0' shape=(9, 1) dtype=float32>], [<tf.Tensor 'while/Identity_58:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_59:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_60:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_61:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_62:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_63:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_64:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_65:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_66:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_67:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_68:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_69:0' shape=(7, 10) dtype=float32>, <tf.Tensor 'while/Identity_70:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/Identity_71:0' shape=(7, 1) dtype=float32>], [<tf.Tensor 'while/Identity_72:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_73:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_74:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_75:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_76:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_77:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_78:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_79:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_80:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_81:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_82:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_83:0' shape=(5, 10) dtype=float32>, <tf.Tensor 'while/Identity_84:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/Identity_85:0' shape=(5, 1) dtype=float32>], [<tf.Tensor 'while/Identity_86:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_87:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_88:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_89:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_90:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_91:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_92:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_93:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_94:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_95:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_96:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_97:0' shape=(13, 10) dtype=float32>, <tf.Tensor 'while/Identity_98:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/Identity_99:0' shape=(13, 1) dtype=float32>], [<tf.Tensor 'while/Identity_100:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_101:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_102:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_103:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_104:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_105:0' shape=(1, 20) dtype=float32>, <tf.Tensor 'while/Identity_106:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_107:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_108:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_109:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_110:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_111:0' shape=(12, 10) dtype=float32>, <tf.Tensor 'while/Identity_112:0' shape=(12, 1) dtype=float32>, <tf.Tensor 'while/Identity_113:0' shape=(12, 1) dtype=float32>]], [<tf.Tensor 'while/Identity_114:0' shape=(1, 20) dtype=float32>], <tf.Tensor 'while/Identity_115:0' shape=(?,) dtype=float32>, <tf.Tensor 'while/Identity_116:0' shape=() dtype=float32>, <tf.Tensor 'while/Identity_117:0' shape= dtype=float32>, <tf.Tensor 'while/Identity_118:0' shape= dtype=int32>, <tf.Tensor 'while/Identity_119:0' shape= dtype=int32>]

Second structure (23 elements): [<tf.Tensor 'while/add_21:0' shape=() dtype=int32>, <tf.Tensor 'while/Add:0' shape=() dtype=float32>, [<tf.Tensor 'while/LOL/PerTensor/sub_15:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_1/sub_15:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_2/sub_15:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_3/sub_15:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_4/sub_15:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_5/sub_15:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_6/sub_15:0' shape=(12, 1) dtype=float32>], [<tf.Tensor 'while/LOL/PerTensor/sub_15:0' shape=(11, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_1/sub_15:0' shape=(3, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_2/sub_15:0' shape=(9, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_3/sub_15:0' shape=(7, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_4/sub_15:0' shape=(5, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_5/sub_15:0' shape=(13, 1) dtype=float32>, <tf.Tensor 'while/LOL/PerTensor_6/sub_15:0' shape=(12, 1) dtype=float32>], <map object at 0x7f0a9b8f4be0>, [<tf.Tensor 'while/LOL/Layer2_RNN/BiasGRUCell/add:0' shape=(1, 20) dtype=float32>], <tf.Tensor 'while/concat:0' shape=(?,) dtype=float32>, <tf.Tensor 'while/Identity_116:0' shape=() dtype=float32>, <tf.Tensor 'while/Identity_117:0' shape= dtype=float32>, <tf.Tensor 'while/Identity_118:0' shape= dtype=int32>, <tf.Tensor 'while/Identity_119:0' shape= dtype=int32>]

qlzh727 commented 6 years ago

Assigning to the owner of the code @olganw and @nirum

ethancaballero commented 6 years ago

bump, I have identical issue

tensorflowbutler commented 4 years ago

Hi There, We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

loremelis commented 4 years ago

I have the same issue, someone have the solution

nightlessbaron commented 4 years ago

I am also facing a similar issue, the problem is coming on Line 347 in trainable_optimizer.py file. It is because in tf.while_loop, the number of elements in loop_body is not equal to that in loop_vars.

Solution : The code is Python 2.7.x compatible, it works fine on this version. In my case, this error was coming as I was using Python 3.x version

nirum commented 4 years ago

Hi, unfortunately, we are not maintaining this code. It’s pretty old at this point, and lots of things have changed in the TensorFlow ecosystem in the meantime.

As you can imagine, the effort required to keep all research projects up to date all the time would quickly subsume our ability to do new research. As such, we are leaving it in the repository as reference, but we aren’t going to maintain it.

tensorflow / models