Input arrays differing dimensions

[DryFrost]

Having issues running the semantic_segmentation_tool.py All file locations are correct however when running the script I get 'ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 1 has 4 dimension(s)'.

I checked the example _rgb.png and it looks like it only has the three channels.

Full Error:

(dl4cv) F:\deepplantphenomics\examples>nano semantic_segmentation_tool.py (dl4cv) F:\deepplantphenomics\examples>python semantic_segmentation_tool.py 2019-10-08 21:41:36.599464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll Performing segmentation... WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:195: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2019-10-08 21:41:39.034906: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll 2019-10-08 21:41:39.182886: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce RTX 2070 major: 7 minor: 5 memoryClockRate(GHz): 1.62 pciBusID: 0000:03:00.0 2019-10-08 21:41:39.190076: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll 2019-10-08 21:41:39.203686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll 2019-10-08 21:41:39.215906: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_100.dll 2019-10-08 21:41:39.227219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_100.dll 2019-10-08 21:41:39.241056: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_100.dll 2019-10-08 21:41:39.253723: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_100.dll 2019-10-08 21:41:39.278110: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll 2019-10-08 21:41:39.284197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2019-10-08 21:41:39.288513: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 2019-10-08 21:41:39.301213: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce RTX 2070 major: 7 minor: 5 memoryClockRate(GHz): 1.62 pciBusID: 0000:03:00.0 2019-10-08 21:41:39.312740: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll 2019-10-08 21:41:39.322045: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll 2019-10-08 21:41:39.326896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_100.dll 2019-10-08 21:41:39.336340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_100.dll 2019-10-08 21:41:39.341408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_100.dll 2019-10-08 21:41:39.351860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_100.dll 2019-10-08 21:41:39.361200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll 2019-10-08 21:41:39.367000: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2019-10-08 21:41:40.089071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-10-08 21:41:40.094314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2019-10-08 21:41:40.097433: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2019-10-08 21:41:40.105783: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6308 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070, pci bus id: 0000:03:00.0, compute capability: 7.5) WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:2043: string_input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced bytf.data. Usetf.data.Dataset.from_tensor_slices(string_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). Ifshuffle=False, omit the.shuffle(...). WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\tensorflow_core\python\training\input.py:277: input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced bytf.data. Usetf.data.Dataset.from_tensor_slices(input_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). Ifshuffle=False, omit the.shuffle(...). WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\tensorflow_core\python\training\input.py:189: limit_epochs (from tensorflow.python.training.input) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced bytf.data. Usetf.data.Dataset.from_tensors(tensor).repeat(num_epochs). WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\tensorflow_core\python\training\input.py:198: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use thetf.datamodule. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\tensorflow_core\python\training\input.py:198: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use thetf.datamodule. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:2045: WholeFileReader.__init__ (from tensorflow.python.ops.io_ops) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced bytf.data. Usetf.data.Dataset.map(tf.read_file). WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:2117: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\tensorflow_core\python\ops\image_ops_impl.py:1518: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\semantic_segmentation_model.py:257: batch (from tensorflow.python.training.input) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced bytf.data. Usetf.data.Dataset.batch(batch_size)(orpadded_batch(...)ifdynamic_pad=True). WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\layers.py:36: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead. WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:910: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:910: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. WARNING:tensorflow:From C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\deepplantpheno.py:203: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use thetf.datamodule. 2019-10-08 21:41:41.950395: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll 2019-10-08 21:41:43.603376: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: Invoking ptxas not supported on Windows Relying on driver to perform ptx compilation. This message will be only logged once. 2019-10-08 21:41:43.660530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll Traceback (most recent call last): File "semantic_segmentation_tool.py", line 20, in <module> y = dpp.tools.segment_vegetation(images) File "C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\tools.py", line 34, in segment_vegetation predictions = net.forward_pass(x) File "C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\networks.py", line 162, in forward_pass y = self.model.forward_pass_with_file_inputs(x) File "C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\deepplantphenomics\semantic_segmentation_model.py", line 311, in forward_pass_with_file_inputs total_outputs = np.append(total_outputs, img, axis=0) File "<__array_function__ internals>", line 6, in append File "C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\numpy\lib\function_base.py", line 4700, in append return concatenate((arr, values), axis=axis) File "<__array_function__ internals>", line 6, in concatenate ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 1 has 4 dimension(s)

Here is pip freeze

`absl-py==0.8.0 alabaster==0.7.12 astor==0.8.0 atomicwrites==1.3.0 attrs==19.2.0 Babel==2.7.0 backcall==0.1.0 beautifulsoup4==4.8.1 bleach==3.1.0 certifi==2019.9.11 chardet==3.0.4 cloudpickle==1.2.2 colorama==0.4.1 cycler==0.10.0 Cython==0.29.13 cytoolz==0.10.0 dask==2.5.2 decorator==4.4.0 deepplantphenomics==0.0.0 defusedxml==0.6.0 docutils==0.15.2 entrypoints==0.3 gast==0.2.2 google-pasta==0.1.7 grpcio==1.24.1 h5py==2.10.0 idna==2.8 imageio==2.6.0 imagesize==1.1.0 imgaug==0.3.0 importlib-metadata==0.23 imutils==0.5.3 ipykernel==5.1.2 ipyparallel==6.2.4 ipython==7.8.0 ipython-genutils==0.2.0 ipywidgets==7.5.1 jedi==0.15.1 Jinja2==2.10.3 joblib==0.14.0 jsonschema==3.0.2 jupyter-client==5.3.3 jupyter-core==4.5.0 Keras==2.2.5 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 kiwisolver==1.1.0 Markdown==3.1.1 MarkupSafe==1.1.1 matplotlib==3.1.1 mistune==0.8.4 mkl-service==2.3.0 mock==3.0.5 more-itertools==7.2.0 nbconvert==5.6.0 nbformat==4.4.0 networkx==2.3 nose==1.3.7 notebook==6.0.1 numpy==1.17.2 olefile==0.46 opencv-contrib-python==4.1.1.26 opencv-python==4.1.1.26 opencv-python-headless==4.1.1.26 opt-einsum==3.1.0 packaging==19.2 pandocfilters==1.4.2 parso==0.5.1 pickleshare==0.7.5 Pillow==6.2.0 pluggy==0.13.0 prometheus-client==0.7.1 prompt-toolkit==2.0.10 protobuf==3.10.0 py==1.8.0 pycocotools==2.0 Pygments==2.4.2 pyparsing==2.4.2 pyrsistent==0.15.4 pytest==5.2.1 python-dateutil==2.8.0 pytz==2019.3 PyWavelets==1.0.3 pywin32==225 pywinpty==0.5.5 PyYAML==5.1.2 pyzmq==18.1.0 qtconsole==4.5.5 requests==2.22.0 scikit-image==0.15.0 scikit-learn==0.21.3 scipy==1.3.1 Send2Trash==1.5.0 Shapely==1.6.4.post2 six==1.12.0 snowballstemmer==2.0.0 soupsieve==1.9.4 Sphinx==2.2.0 sphinxcontrib-applehelp==1.0.1 sphinxcontrib-devhelp==1.0.1 sphinxcontrib-htmlhelp==1.0.2 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.2 sphinxcontrib-serializinghtml==1.1.3 tb-nightly==2.1.0a20191007 tensorboard==1.15.0 tensorflow-estimator==1.15.1 tensorflow-gpu==1.15.0rc3 termcolor==1.1.0 terminado==0.8.2 testpath==0.4.2 toolz==0.10.0 tornado==6.0.3 tqdm==4.36.1 traitlets==4.3.3 urllib3==1.25.6 wcwidth==0.1.7 webencodings==0.5.1 Werkzeug==0.16.0 widgetsnbextension==3.5.1 wincertstore==0.2 wrapt==1.11.2 zipp==0.6.0 `

[jubbens]

@donovanlavoie can you look into this issue on master?

[donovanlavoie]

That error seems to have been coming from the guts of the semantic segmentation model object, so you weren't doing anything wrong. It boiled down to fixing a small bug in how the forward pass collected its outputs, and making the segmentation tool's thresholding see the same size input as it was before.

The segmentation tool should be running fine again after this. Let us know if another problem occurs.

[DryFrost]

New error File "semantic_segmentation_tool.py", line 30, in <module> mask_img = Image.fromarray((img * 255).astype(np.uint8)) File "C:\Users\Dane Nguyen\Anaconda3\envs\dl4cv\lib\site-packages\PIL\Image.py", line 2659, in fromarray size = shape[1], shape[0] IndexError: tuple index out of range

[donovanlavoie]

Is it safe to assume that you're on the master branch?

I thought that error would have been dealt with already, so something else must be going on as well. I suspect that a call to cv2.threshold in the segment_vegetation tool is (still) misinterpreting the raw predictions given by the semantic segmentation model.

The problem is that I can't currently reproduce the issue again. @jubbens thinks that maybe your images somehow have an alpha channel (though unlikely if you confirmed that your images have 3 channels).

Is there any chance that your using images from a public dataset? That way I could get the images myself and keep trying to reproduce the problem.

[DryFrost]

Yes. I pulled the latest master branch. The images I'm using is located in deepplantphenomics/deepplantphenomics/tests/test_data/test_Ara2013_Canon/

[donovanlavoie]

I could reproduce the issue in one case: passing only 1 image to the segmentation tool with the previous fix would result in overzealous dimension pruning. I also made thresholding happen one image at a time in order to try and stem any potential issues cv2.threshold may be causing. If you pull the new commit from master and try again, does the issue persist?

[DryFrost]

Works as expected.