search

xtudbxk / DSRG-tensorflow

a tensorflow version for DSRG (Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing)
75 stars 20 forks source link

number of elements does not match. #17

Closed NanWangAC closed 5 years ago

NanWangAC commented 5 years ago

when I run the project something wrong happened, information as follows:

`tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,3], [batch]: [321,321,1] [[node IteratorGetNext (defined at /workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py:109) ]]

Errors may have originated from an input operation. Input Source operations connected to node IteratorGetNext: IteratorV2 (defined at /workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py:108)

Original stack trace for 'IteratorGetNext': File "/workbench/DSRG-tensorflow-master/DSRG.py", line 409, in dsrg.train(base_lr=lr,weight_decay=5e-4,momentum=0.9,batch_size=batch_size,epoches=epoches) File "/workbench/DSRG-tensorflow-master/DSRG.py", line 349, in train data_x,data_y,data_tag,data_cues,id_of_image,iterator_train = self.data.next_batch(category="train",batch_size=batch_size,epoches=-1) File "/workbench/DSRG-tensorflow-master/pythonlib/dataset_DSRG.py", line 109, in nextbatch img,gt,tag,cues,id = iterator.get_next() File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 426, in get_next output_shapes=self._structure._flat_shapes, name=name) File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 1947, in iterator_get_next output_shapes=output_shapes, name=name) File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(*args, **kwargs) File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op op_def=op_def) File "/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2005, in init self._traceback = tf_stack.extract_stack() ` what should i do？ thank u

xtudbxk commented 5 years ago

It seems the version of tensorflow doesn't match. In my project, it is 1.14.0. And you can check it by python -c "import tensorflow as tf;print(tf.__version__)"

NanWangAC commented 5 years ago

oh, I think it may be data problem...do you have any other data download link?

xtudbxk commented 5 years ago

The download link I provided is from the official website. Maybe you can download again?

NanWangAC commented 5 years ago

What i need download are SBD and 2012voc?

xtudbxk commented 5 years ago

Yes. As mentioned in the paper, the complete data set has two set. One is the PASCAL VOC 2012 and the other is SBD.

NanWangAC commented 5 years ago

I will try again.but it download slowly... thank u

xtudbxk commented 5 years ago

Ok, you are welcome. Just remeber to run the convert.py in data folder to merge those two sets.

NanWangAC commented 5 years ago

when i run convert.py some img generated is totally black,some img are broken ,this is the reason why number of elements does not match.What should I do?

NanWangAC commented 5 years ago

And the broken imgs both in SBD and voc 2012, correctly imgs just only in SBD or VOC2012, Maybe rewrite break the imgs?

xtudbxk commented 5 years ago
  1. The convert.py just extracts the labels from SBD dataset. So the output imgs contain the category index such as 0,1,2... And it is normal that the imgs are black in our eyes.
  2. What are the broken images in your problem?
  3. And do you means that the network works normally in sepearted SBD or VOC2012 set and fails under the merged set in your experiments? 不然你直接说中文吧，我英文不太好。。。
NanWangAC commented 5 years ago

就是SBD中有些数据从mat转换成png时，png的内容是一片黑什么都没有，训练时读到这部分就会报错。

xtudbxk commented 5 years ago

全黑是正常现象吧，这里是将SBD数据集的label给提取出来。由于类别是从0到20的，所以保存后人眼是看起来是全黑的，但实际读取是正常的。你可以手动载入SBD和VOC2012中的一张label对比下，看下应该是类似的。

NanWangAC commented 5 years ago

分别跑VOC和SBD的时候报的错都是维度不匹配，但是细节却是反过来的 voc: [tensor]:[321,321,1], [batch]: [321,321,3] tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,1], [batch]: [321,321,3] [[{{node IteratorGetNext}}]] [[IteratorGetNext/_106]] (1) Invalid argument: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,1], [batch]: [321,321,3] [[{{node IteratorGetNext}}]]

SBD: [tensor]: [321,321,3], [batch]: [321,321,1] tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,3], [batch]: [321,321,1] [[{{node IteratorGetNext}}]] (1) Invalid argument: Cannot add tensor to the batch: number of elements does not match. Shapes are: [tensor]: [321,321,3], [batch]: [321,321,1] [[{{node IteratorGetNext}}]] [[vgg/PyFunc/_137]]

xtudbxk commented 5 years ago

如果你想要更详细的分析，可能你需要提供更多的信息给我，比如

  1. 这个是一开始就报错，还是跑了一会儿，然后才报的错？(这有助于判断是部分数据有问题，还是全部数据有问题，亦或是代码有问题)
  2. 你能pythonlib/dataset_DSRG.py的get_data函数中(约86行)添加 print(identy)嘛? (这有助于判断是构建图时的问题，还是执行图时的问题，并得到出错的图片的id)
NanWangAC commented 5 years ago

要不加个微信或者QQ？卡在这里有段时间了

xtudbxk commented 5 years ago

可以呀，你发我邮件吧 xtudbxk@126.com

HeliumTang commented 5 years ago

可以呀，你发我邮件吧 xtudbxk@126.com

作者你好。我也遇到同样的问题了。 请问应该怎么解决？

xtudbxk commented 5 years ago

他好像是数据有问题，重下就好了

xtudbxk

邮箱：xtudbxk@126.com |

签名由 网易邮箱大师 定制

On 10/25/2019 17:30, HeliumTang wrote:

可以呀，你发我邮件吧 xtudbxk@126.com

作者你好。我也遇到同样的问题了。 请问应该怎么解决？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

HeliumTang commented 5 years ago

他好像是数据有问题，重下就好了 | | xtudbxk | | 邮箱：xtudbxk@126.com | 签名由 网易邮箱大师 定制 On 10/25/2019 17:30, HeliumTang wrote: 可以呀，你发我邮件吧 xtudbxk@126.com 作者你好。我也遇到同样的问题了。 请问应该怎么解决？ — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

在convert.py运行时出现了这个，是正确的吗？ python3.7/site-packages/skimage/io/_io.py:141: UserWarning: VOCdevkit/VOC2012/SegmentationClassAug/2010_004327.png is a low contrast image warn('%s is a low contrast image' % fname)

HeliumTang commented 5 years ago

他好像是数据有问题，重下就好了 | | xtudbxk | | 邮箱：xtudbxk@126.com | 签名由 网易邮箱大师 定制 On 10/25/2019 17:30, HeliumTang wrote: 可以呀，你发我邮件吧 xtudbxk@126.com 作者你好。我也遇到同样的问题了。 请问应该怎么解决？ — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

我数据重新下载了也还是不行

HeliumTang commented 5 years ago

就是SBD中有些数据从mat转换成png时，png的内容是一片黑什么都没有，训练时读到这部分就会报错。

请问你是怎么解决这个问题的

xtudbxk commented 5 years ago
  1. convert.py那个提示不用管，不会影响结果
  2. 是所有数据都重下了嘛？ 训练集数据由PASCAL VOC 2012和SBD两个数据集合成的
  3. 一片黑并非有问题，而是由于类别的编码是从 0到20, 所以值较小，在png图中显示出来就是一片黑。而使用库读出图片的具体数值，并没有问题
HeliumTang commented 5 years ago
  1. convert.py那个提示不用管，不会影响结果
  2. 是所有数据都重下了嘛？ 训练集数据由PASCAL VOC 2012和SBD两个数据集合成的
  3. 一片黑并非有问题，而是由于类别的编码是从 0到20, 所以值较小，在png图中显示出来就是一片黑。而使用库读出图片的具体数值，并没有问题

完全不行哎， 我全部重新弄了一遍， 还是不行。。我等他的回复好了

xtudbxk commented 5 years ago

重新翻了下聊天记录，他是把batch_size改为1之后绕过了这问题的。。。

HeliumTang commented 5 years ago

重新翻了下聊天记录，他是把batch_size改为1之后绕过了这问题的。。。

请问你训练的时候用了几张GPU？