search

moabitcoin / holy-edge

Holistically-Nested Edge Detection
https://arxiv.org/pdf/1504.06375.pdf
GNU General Public License v3.0
345 stars 136 forks source link

Removing resizing image step #9

Open priyanka-chaudhary opened 6 years ago

priyanka-chaudhary commented 6 years ago

From the HED paper I understood that we don't need to resize images as the network doesn't have any fully connected layers. So for my own dataset I wanted to change your code to remove this step so it take any size image and also produces edge map of same size as image.

But just removing these lines

im = im.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height'])) em = em.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height'])) is giving an error.

Is it possible to do this with your code? Any hint or pointer would be appreciated. Thank you.

sandhawalia commented 6 years ago

Hi, Could you please copy paste the error you are getting here ?

Yes indeed HED doesnt fixed size image for training. Image resizing is done for faster training in this case since batch processing comes in handy with resized images. One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time).

priyanka-chaudhary commented 6 years ago

I commented these lines in data_parser.py

im = im.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height'])) em = em.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height']))

And try to train I get the following error:

pchaudha@costa:~/hed$ python run-hed.py --train --gpu-limit=0.7 --config-file='/home/pchaudha/hed/hed/configs/hed.yaml' 2017-12-21 16:44:16.193934: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-12-21 16:44:16.193956: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-12-21 16:44:16.193961: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. 2017-12-21 16:44:16.193964: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations. 2017-12-21 16:44:16.193968: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations. 2017-12-21 16:44:16.425315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate (GHz) 1.076 pciBusID 0000:01:00.0 Total memory: 11.91GiB Free memory: 11.41GiB 2017-12-21 16:44:16.425345: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 2017-12-21 16:44:16.425350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y 2017-12-21 16:44:16.425361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0) [21 Dec 2017 16h44m16s][INFO] Model weights loaded from vgg16.npy [21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-1+SIDE-1 [21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-2+SIDE-2 [21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-3+SIDE-3 [21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-4+SIDE-4 [21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-5+SIDE-5 [21 Dec 2017 16h44m16s][INFO] Added FUSE layer [21 Dec 2017 16h44m16s][INFO] Build model finished: 0.1302s [21 Dec 2017 16h44m16s][INFO] Done initializing VGG-16 model [21 Dec 2017 16h44m16s][INFO] Training data set-up from /home/pchaudha/hed/hed-data/HED-BSDS/train_pair.lst [21 Dec 2017 16h44m16s][INFO] Training samples 23040 [21 Dec 2017 16h44m16s][INFO] Validation samples 5760 [21 Dec 2017 16h44m16s][WARNING] Deep supervision application set to True Traceback (most recent call last): File "run-hed.py", line 64, in main(args) File "run-hed.py", line 38, in main trainer.run(session) File "/home/pchaudha/hed/hed/train.py", line 69, in run run_metadata=run_metadata) File "/home/pchaudha/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run run_metadata_ptr) File "/home/pchaudha/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1093, in _run np_val = np.asarray(subfeed_val, dtype=subfeed_dtype) File "/home/pchaudha/.local/lib/python2.7/site-packages/numpy/core/numeric.py", line 531, in asarray return array(a, dtype, copy=False, order=order) ValueError: setting an array element with a sequence.

For this part of your comment: " One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time)." What should be written for testing image_width and image_height in hed.yaml file then?

sandhawalia commented 6 years ago

For this part of your comment: " One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time)." What should be written for testing image_width and image_height in hed.yaml file then?

You can set it to large value like 3200. This would work only if the test images are in a batch size of 1 though.

File "/home/pchaudha/.local/lib/python2.7/site-packages/numpy/core/numeric.py", line 531, in asarray return array(a, dtype, copy=False, order=order) ValueError: setting an array element with a sequence.

This part is due to the fact that all training image are of different sizes and cant be processed a as batch on GPU. A way around would be reduce batch size to 1 and disabling image width and height ( Disclaimer : i haven't tried it myself)

priyanka-chaudhary commented 6 years ago

"You can set it to large value like 3200. This would work only if the test images are in a batch size of 1 though." I tried this and it works with 3200. But when I try another value it doesn't. Can you please give some insight why? Like when I tried with 1200 it gives error even with batch size 1. I know it sounds stupid but I am not able to figure it out.

"This part is due to the fact that all training image are of different sizes and cant be processed a as batch on GPU. A way around would be reduce batch size to 1 and disabling image width and height ( Disclaimer : i haven't tried it myself)" Also I wanted to try this so how should I go about it? Like changing batch_size_train to 1 I got. What should I put for image_width and image_height?

Thanks a lot for the support. Really appreciate it.

yfnn commented 6 years ago

I also have the same question. Now your code give outputs of fixed size (480*320). But I want to get flexible output size. I hope the output size is the same with the input size. So, I change the some code. I change image placeholder's size into [None,None,None,3], and edgemap placeholder's size into [None,None,None,1]. Then, in test.py, I ignore the image resize line. But, I get this error. Could you give me some help?

2018-06-05 03:47:51.917737: W tensorflow/core/framework/op_kernel.cc:1152] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1] [[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]] Traceback (most recent call last): File "run-hed.py", line 55, in parser = argparse.ArgumentParser(description='Utility for Training/Testing DL models(Concepts/Captions) using theano/keras') File "run-hed.py", line 44, in main tester.run(session) File "/home/fayan/holy-edge/hed/test.py", line 68, in run edgemap = session.run(self.model.predictions, feed_dict={self.model.images: [im]}) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 778, in run run_metadata_ptr) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 983, in _run feed_dict_string, options, run_metadata) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1033, in _do_run target_list, options, run_metadata) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1053, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1] [[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]] [[Node: output_5/_45 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_23_output_5", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Caused by op u'concat', defined at: File "run-hed.py", line 55, in parser = argparse.ArgumentParser(description='Utility for Training/Testing DL models(Concepts/Captions) using theano/keras') File "run-hed.py", line 43, in main tester.setup(session) File "/home/fayan/holy-edge/hed/test.py", line 37, in setup self.model = Vgg16(self.cfgs, run='testing') File "/home/fayan/holy-edge/hed/models/vgg16.py", line 33, in init self.define_model() File "/home/fayan/holy-edge/hed/models/vgg16.py", line 84, in define_model self.fuse = self.conv_layer(tf.concat(self.side_outputs, axis=3), File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1034, in concat name=name) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 519, in _concat_v2 name=name) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op op_def=op_def) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op original_op=self._default_original_op, op_def=op_def) File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1228, in init self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1] [[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]] [[Node: output_5/_45 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_23_output_5", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]