search

warmspringwinds / tf-image-segmentation

Image Segmentation framework based on Tensorflow and TF-Slim library
MIT License
549 stars 188 forks source link

Error during training in training.py #23

Closed ghost closed 7 years ago

ghost commented 7 years ago

while training I am getting error in line valid_logits_batch_tensor = tf.gather_nd(params=logits_batch_tensor, indices=valid_batch_indices) in function training.py

Error:

InvalidArgumentError (see above for traceback): Invalid indices: [31,1] = [0, 76, 192] does not index into [1,192,192,21] [[Node: adam_vars/gradients/GatherNd_1_grad/ScatterNd = ScatterNd[T=DT_FLOAT, Tindices=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](ToInt32_3/_587, adam_vars/gradients/Reshape_4_grad/Reshape/_589, adam_vars/gradients/GatherNd_1_grad/Shape/_591)]] [[Node: adam_vars/gradients/GatherNd_1_grad/ScatterNd/_593 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_252_adam_vars/gradients/GatherNd_1_grad/ScatterNd", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

Training code

Did any of you faced similar issue ?

ghost commented 7 years ago

I was using tf 1.0 version, after upgrading to tf 1.1 the issue is resolved.