search

matpalm / bnn

bee detection tensorflow conv net for a rasp pi on side of a hive
http://matpalm.com/blog/counting_bees/
MIT License
405 stars 67 forks source link

ValueError: A `Concatenate` layer requires inputs with matching shapes ... #23

Closed lamenace91 closed 5 years ago

lamenace91 commented 5 years ago

Hi,

I'm trying to use bnn to count bees but I have an error at the training step. I have one image file in training and test directories. Is there something wrong with my image ? [picture,](url timelapse-20190603-133222 jpg_resized ) Thank you for your help, Loic

PS: let me know if you need the output of the previous steps

I ran: 

./label_ui.py \
    --image-dir sample_data/training/ \
    --label-db sample_data/labels.db \
    --width 1365 --height 1024
./materialise_label_db.py \
    --label-db sample_data/labels.db \
    --directory sample_data/labels/ \
    --width 1365 --height 1024
./data.py \
    --image-dir sample_data/training/ \
    --label-dir sample_data/labels/ \
    --rotate --distort \
    --patch-width-height 256
./train.py \
    --run r12 \
    --steps 2 \
    --train-steps 2 \
    --batch-size 4 \
    --train-image-dir sample_data/training/ \
    --test-image-dir sample_data/test/ \
    --label-dir sample_data/labels/ \
    --pos-weight 5 \
    --patch-width-height 256 \
    --width 1365 --height 1024

The output of train.py is:

opts Namespace(base_filter_size=8, batch_size=4, connected_components_threshold=0.05, flip_left_right=False, height=1024, label_db='label.201802_sample.db', label_dir='sample_data/labels/', learning_rate=0.001, no_use_batch_norm=False, no_use_skip_connections=False, patch_width_height=256, pos_weight=5.0, random_rotate=False, run='r12', secs=None, steps=2, test_image_dir='sample_data/test/', train_image_dir='sample_data/training/', train_steps=2, width=1365)
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:423: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
len(rgb_filenames) 1 CACHE

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

num_test_files= 1 batch_size= 4 => num_test_steps= 0
TRAIN MODEL
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
inputs (InputLayer)             (None, 256, 256, 3)  0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 128, 128, 8)  224         inputs[0][0]                     
__________________________________________________________________________________________________
batch_normalization_v1 (BatchNo (None, 128, 128, 8)  32          conv2d[0][0]                     
__________________________________________________________________________________________________
re_lu (ReLU)                    (None, 128, 128, 8)  0           batch_normalization_v1[0][0]     
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 64, 64, 16)   1168        re_lu[0][0]                      
__________________________________________________________________________________________________
batch_normalization_v1_1 (Batch (None, 64, 64, 16)   64          conv2d_1[0][0]                   
__________________________________________________________________________________________________
re_lu_1 (ReLU)                  (None, 64, 64, 16)   0           batch_normalization_v1_1[0][0]   
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 32, 32, 32)   4640        re_lu_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization_v1_2 (Batch (None, 32, 32, 32)   128         conv2d_2[0][0]                   
__________________________________________________________________________________________________
re_lu_2 (ReLU)                  (None, 32, 32, 32)   0           batch_normalization_v1_2[0][0]   
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 16, 16, 64)   18496       re_lu_2[0][0]                    
__________________________________________________________________________________________________
batch_normalization_v1_3 (Batch (None, 16, 16, 64)   256         conv2d_3[0][0]                   
__________________________________________________________________________________________________
re_lu_3 (ReLU)                  (None, 16, 16, 64)   0           batch_normalization_v1_3[0][0]   
__________________________________________________________________________________________________
e4nn (UpSampling2D)             (None, 32, 32, 64)   0           re_lu_3[0][0]                    
__________________________________________________________________________________________________
d1_e3 (Concatenate)             (None, 32, 32, 96)   0           e4nn[0][0]                       
                                                                 re_lu_2[0][0]                    
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 32, 32, 32)   27680       d1_e3[0][0]                      
__________________________________________________________________________________________________
batch_normalization_v1_4 (Batch (None, 32, 32, 32)   128         conv2d_4[0][0]                   
__________________________________________________________________________________________________
re_lu_4 (ReLU)                  (None, 32, 32, 32)   0           batch_normalization_v1_4[0][0]   
__________________________________________________________________________________________________
d1nn (UpSampling2D)             (None, 64, 64, 32)   0           re_lu_4[0][0]                    
__________________________________________________________________________________________________
d2_e2 (Concatenate)             (None, 64, 64, 48)   0           d1nn[0][0]                       
                                                                 re_lu_1[0][0]                    
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 64, 64, 16)   6928        d2_e2[0][0]                      
__________________________________________________________________________________________________
batch_normalization_v1_5 (Batch (None, 64, 64, 16)   64          conv2d_5[0][0]                   
__________________________________________________________________________________________________
re_lu_5 (ReLU)                  (None, 64, 64, 16)   0           batch_normalization_v1_5[0][0]   
__________________________________________________________________________________________________
d2nn (UpSampling2D)             (None, 128, 128, 16) 0           re_lu_5[0][0]                    
__________________________________________________________________________________________________
d3_e1 (Concatenate)             (None, 128, 128, 24) 0           d2nn[0][0]                       
                                                                 re_lu[0][0]                      
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 128, 128, 8)  1736        d3_e1[0][0]                      
__________________________________________________________________________________________________
batch_normalization_v1_6 (Batch (None, 128, 128, 8)  32          conv2d_6[0][0]                   
__________________________________________________________________________________________________
re_lu_6 (ReLU)                  (None, 128, 128, 8)  0           batch_normalization_v1_6[0][0]   
__________________________________________________________________________________________________
logits (Conv2D)                 (None, 128, 128, 1)  9           re_lu_6[0][0]                    
==================================================================================================
Total params: 61,585
Trainable params: 61,233
Non-trainable params: 352
__________________________________________________________________________________________________
None
Traceback (most recent call last):
  File "./train.py", line 103, in <module>
    use_batch_norm=not opts.no_use_batch_norm)
  File "/home/ponger/Apis/monitor/toBorNot2B/bnn/model.py", line 57, in construct_model
    d1 = Concatenate(name='d1_e3')([d1, e3])
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 538, in __call__
    self._maybe_build(inputs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 1603, in _maybe_build
    self.build(input_shapes)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/utils/tf_utils.py", line 151, in wrapper
    output_shape = fn(instance, input_shape)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/layers/merge.py", line 392, in build
    'Got inputs shapes: %s' % (input_shape))
ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 128, 172, 64), (None, 128, 171, 32)]
matpalm commented 5 years ago

yeah that's to do with the skip connections not aligning in size. i was just lazily supporting the images i have which have an exact ratio of 4/3. couple of things you could do...

  1. disable the skip connections (use --no-use-skip-connections where it's expected). this might just work & i only saw marginal improvement from it anyways.
  2. resize your images to be exactly 4/3 ratio; e.g. from --width 1365 --height 1024 to --width 1024 --height 768
  3. fix the code to be more robust to the size difference; (e4nn and e3 currently have different spatial sizes)
lamenace91 commented 5 years ago

It works by modifying the aspect ratio. I will try the first solution a little bit later. Great, thanks!