ValueError: No variables provided generated by opt.apply_gradients(grads_and_vars, global_step=global_step)

[wenouyang]

Hi Marvin,

Since I was not able to get the tf.add_n(tf.get_collection(reg_loss_col)) work, I just remove weight_loss from total_loss in kitti_mutliloss.py

total_loss = cross_entropy_mean #+ weight_loss

Then running the program generates the following error,

File "/home/GPU-Study/keras/FCN/kittiseg/tensorvision/train.py", line 386, in do_training
    tv_graph = core.build_training_graph(hypes, queue, modules)
  File "/home/GPU-Study/keras/FCN/kittiseg/tensorvision/core.py", line 100, in build_training_graph
    global_step, learning_rate)
  File "/home/GPU-Study/keras/FCN/kittiseg/hypes/../optimizer/generic_optimizer.py", line 94, in training
    train_op = opt.apply_gradients(grads_and_vars, global_step=global_step)
  File "/devl/tensorflow/tf/lib/python3.4/site-packages/tensorflow/python/training/optimizer.py", line 370, in apply_gradients
    raise ValueError("No variables provided.")
ValueError: No variables provided.

I then printed out tuple(grads_and_vars), which is shown as following. What kind of information it provides, and why it can raise the valueerror of "no variables provided".

-----------------tuple(grads_and_vars)----------------------- ((<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_0:0' shape=(3, 3, 3, 64) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b6d7646fbe0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_1:0' shape=(64,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4822cc0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_2:0' shape=(3, 3, 64, 64) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b6d7646fb38>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_3:0' shape=(64,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b6d7646feb8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_4:0' shape=(3, 3, 64, 128) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4813940>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_5:0' shape=(128,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e48e0c18>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_6:0' shape=(3, 3, 128, 128) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e486da58>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_7:0' shape=(128,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e480b828>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_8:0' shape=(3, 3, 128, 256) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e499a518>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_9:0' shape=(256,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e49e3128>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_10:0' shape=(3, 3, 256, 256) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e49f4ac8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_11:0' shape=(256,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4a48fd0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_12:0' shape=(3, 3, 256, 256) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4a579b0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_13:0' shape=(256,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4ab5dd8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_14:0' shape=(3, 3, 256, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4b31240>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_15:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4b61cc0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_16:0' shape=(3, 3, 512, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4b38cf8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_17:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4bc40b8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_18:0' shape=(3, 3, 512, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4c17f28>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_19:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4bf6f98>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_20:0' shape=(3, 3, 512, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4c95e80>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_21:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4ca4198>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_22:0' shape=(3, 3, 512, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4cd55f8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_23:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4b48278>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_24:0' shape=(3, 3, 512, 512) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4ccc048>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_25:0' shape=(512,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4dac828>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_26:0' shape=(7, 7, 512, 4096) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4d8cef0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_27:0' shape=(4096,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4d5ae80>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_28:0' shape=(1, 1, 4096, 4096) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4e8e208>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_29:0' shape=(4096,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4e8e470>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_30:0' shape=(1, 1, 4096, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4f11da0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_31:0' shape=(2,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4f30ba8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_32:0' shape=(4, 4, 2, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4f93f28>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_33:0' shape=(1, 1, 512, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e4fe45c0>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_34:0' shape=(2,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e50063c8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_35:0' shape=(4, 4, 2, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e5073550>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_36:0' shape=(1, 1, 256, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e50b13c8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_37:0' shape=(2,) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e50dc9e8>), (<tf.Tensor 'Optimizer/training/clip_by_global_norm/Optimizer/training/clip_by_global_norm/_38:0' shape=(16, 16, 2, 2) dtype=float32>, <tensorflow.python.ops.variables.Variable object at 0x2b70e50ceda0>))

[MarvinTeichmann]

I have no idea, where this error comes from :(. I did never came across it myself. It looks like loss is well defined, so there should not be an error.

[MarvinTeichmann]

Another possibility might be some recent tensorflow api changes. I haven't done any tf update since 1.0. Maybe try the first tf 1.0 version?

[MarvinTeichmann]

I think this is a python 3.5 error. Currently only python 2.7 seams to work. Also see #35.

[avitalsh]

Hi Marvin, I got the same "No variables provided." error as mentioned above, when running with python3.5. Is this an issue that can be solved? I can't use python2.7.. Thanks

[mohbattharani]

I have "No variables provided." issue with Python3.5 (KittiSeg Training). Is there any way to solve it on 3.5? Or did anyone get this issue resolved by using 2.7?

[kalanityL]

@mohbattharani yes, solved by using 2.7

(in my case it was the tensorvision that I installed with a pip set for 3.5 and tv-continue was then raising this same error.

I reinstalled tensorvision using pip2.7 and this error did not happen again.