Concat Mode Error

[t-schaefer]

Hi,

I still have this error even though I've edited my Keras.json file according to solved issue with the same problem. I got this: (venv) parallels@parallels-Parallels-Virtual-Platform:~/Saliency/sam$ python main.py test Saliency/sam/sample_images Using Theano backend. /home/parallels/venv/lib/python3.6/site-packages/keras/backend/theano_backend.py:1282: UserWarning: DEPRECATION: the 'ds' parameter is not going to exist anymore as it is going to be replaced by the parameter 'ws'. mode='max') /home/parallels/venv/lib/python3.6/site-packages/keras/backend/theano_backend.py:1282: UserWarning: DEPRECATION: the 'st' parameter is not going to exist anymore as it is going to be replaced by the parameter 'stride'. mode='max') /home/parallels/venv/lib/python3.6/site-packages/keras/backend/theano_backend.py:1282: UserWarning: DEPRECATION: the 'padding' parameter is not going to exist anymore as it is going to be replaced by the parameter 'pad'. mode='max') Traceback (most recent call last): File "main.py", line 63, in m = Model(input=[x, x_maps], output=sam_resnet([x, x_maps])) File "/home/parallels/Saliency/sam/models.py", line 140, in sam_resnet concateneted = merge([att_convlstm, priors1], mode='concat', concat_axis=1) File "/home/parallels/venv/lib/python3.6/site-packages/keras/engine/topology.py", line 1528, in merge name=name) File "/home/parallels/venv/lib/python3.6/site-packages/keras/engine/topology.py", line 1186, in init node_indices, tensor_indices) File "/home/parallels/venv/lib/python3.6/site-packages/keras/engine/topology.py", line 1253, in _arguments_validation 'Layer shapes: %s' % (input_shapes)) Exception: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 512, 60, 80), (None, 16, 30, 40)]

Have you got any ideas what the problem there is? Should I try a later version of Theano? Thank you very much.

[marcellacornia]

Hi @t-schaefer, thanks for downloading our code.

Which Keras and Theano versions are you using?

The error seems related to the input image size. Have you changed the default parameters in the config.py file?

[t-schaefer]

Hi Marcella,

thanks for your answer. I am using Keras Version 1.1.0 and Theano Version 0.9. Yes, I think changed the config file accordingly: #########################################################################

MODEL PARAMETERS

#########################################################################

version (0 for SAM-VGG and 1 for SAM-ResNet)

version = 1

batch size

b_s = 1

number of rows of input images

shape_r = 717

number of cols of input images

shape_c = 478

number of rows of downsampled maps

shape_r_gt = 30

number of cols of downsampled maps

shape_c_gt = 40

number of rows of model outputs

shape_r_out = 480

number of cols of model outputs

shape_c_out = 640

final upsampling factor

upsampling_factor = 16

number of epochs

nb_epoch = 10

number of timestep

nb_timestep = 4

number of learned priors

nb_gaussian = 16

#########################################################################

TRAINING SETTINGS

#########################################################################

path of training images

imgs_train_path = '/Saliency/Images-Test1/'

path of training maps

maps_train_path = '/Saliency/training'

path of training fixation maps

fixs_train_path = '/Saliency/fixation_maps/'

number of training images

nb_imgs_train = 10000

path of validation images

imgs_val_path = '/Saliency/validation/images/'

path of validation maps

maps_val_path = '/Saliency/validation/maps/'

path of validation fixation maps

fixs_val_path = '/Saliency/validation/fixation_maps/'

number of validation images

nb_imgs_val = 5000

Am 27.12.2018 um 11:40 schrieb Marcella Cornia notifications@github.com:

Hi @t-schaefer https://github.com/t-schaefer, thanks for downloading our code.

Which Keras and Theano versions are you using?

The error seems related to the input image size. Have you changed the default parameters in the config.py file?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/marcellacornia/sam/issues/20#issuecomment-450126554, or mute the thread https://github.com/notifications/unsubscribe-auth/Ar9B1lDxLlWwNCVXbzCJTQKTnwFxQO7Jks5u9KO4gaJpZM4Zf4W_.

[marcellacornia]

If you just want to use our code in the testing mode (i.e. predict a set of saliency maps without re-training the model), I suggest not to change the default parameters.

Having input images with different sizes is not a problem. Indeed, our code pre-processes all input images by re-sizing them according to the input size used to train the model. Note that the images are resized without losing the original aspect ratio and using zero-padding if necessary. In the end, the predicted saliency maps are restored to the size of the original input images.

[t-schaefer]

That is very good to know. Actually, I tried it without changing the confit file first, because I thought it would be designed for using the test images, but I still got the same error.

Am 27.12.2018 um 12:08 schrieb Marcella Cornia notifications@github.com:

If you just want to use our code in the testing mode (i.e. predict a set of saliency maps without re-training the model), I suggest not to change the default parameters.

Having input images with different sizes is not a problem. Indeed, our code pre-processes all input images by re-sizing them according to the input size used to train the model. Note that the images are resized without losing the original aspect ratio and using zero-padding if necessary. In the end, the predicted saliency maps are restored to the size of the original input images.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[marcellacornia]

It is very strange. The error you reported is related to the concatenation of the output of the attentive lstm module with the first set of learned priors.

The shape of the attentive lstm output should be (batch size, 512, 30, 40). The last two dimensions are obtained from the original image size (i.e. 240x320) divided by a factor of 8, due to max pooling or convolutions with strides greater than one (as in the ResNet model). For this reason, keeping the default configuration parameters should not cause any problems.

The only difference that I noticed is the Python version: in the original implementation, I used Python 2.7. I never tried to run the code with Python 3.6 and I'm not sure if this could cause errors or not.

[dksshddl]

I have same error.

my environment:

• ubuntu 18.04
• python 3.5
• keras 1.1
• theano 0.9

and I didn't change config.py parameter.

I run script "python main.py test sample_images/".

error log:

Traceback (most recent call last): File "main.py", line 63, in m = Model(input=[x, x_maps], output=sam_resnet([x, x_maps])) File "/home/dksshddl/PycharmProjects/sam/models.py", line 140, in sam_resnet concateneted = merge([att_convlstm, priors1], mode='concat', concat_axis=1) File "/home/dksshddl/anaconda3/envs/sam/lib/python3.5/site-packages/keras/engine/topology.py", line 1528, in merge name=name) File "/home/dksshddl/anaconda3/envs/sam/lib/python3.5/site-packages/keras/engine/topology.py", line 1186, in init node_indices, tensor_indices) File "/home/dksshddl/anaconda3/envs/sam/lib/python3.5/site-packages/keras/engine/topology.py", line 1253, in _arguments_validation 'Layer shapes: %s' % (input_shapes)) Exception: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 512, 30, 512), (None, 16, 30, 40)]

[ghost]

+1

[Fanshia]

+1

[marcellacornia]

We noticed that this problem could be due to different versions of the required dependencies. Our code is compatible with Python 2.7, Keras 1.1 and Theano 0.9.

[Fanshia]

I tried Python 2.7, Keras 1.1 and Theano 0.9. Still the same problem.

[marcellacornia]

Have you checked your keras.json file? It should contain "image_dim_ordering": "th" and "backend": "theano".

[Fanshia]

You are right. I changed .keras/keras.json to {
"image_dim_ordering": "th",
"image_data_format": "channels_first",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "theano"
}
This works for me.