standardize_input_data comes to a shape problem

[bury-source]

when I'm training with Recipe5k using InceptionV3 model, it comes to a problem which I cannot fix it.

Traceback (most recent call last):
  File "main.py", line 198, in <module>
    train_model(params)
  File "main.py", line 72, in train_model
    ing_model.trainNet(dataset, training_params)
  File "/foodingre/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 677, in t
rainNet    self.__train(ds, params)
  File "/foodingre/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 862, in _
_train    initial_epoch=params['epoch_offset'])
  File "/foodingre/keras-2.2.4.1/keras-2.2.4.1/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/foodingre/keras-2.2.4.1/keras-2.2.4.1/keras/engine/training.py", line 1470, in fit_generator
    initial_epoch=initial_epoch)
  File "/oodingre/keras-2.2.4.1/keras-2.2.4.1/keras/engine/training_generator.py", line 217, in fit_generator
    class_weight=class_weight)
  File "/foodingre/keras-2.2.4.1/keras-2.2.4.1/keras/engine/training.py", line 1263, in train_on_batch
    class_weight=class_weight)
  File "/foodingre/keras-2.2.4.1/keras-2.2.4.1/keras/engine/training.py", line 763, in _standardize_user_data
    exception_prefix='input')
  File "/foodingre/keras-2.2.4.1/keras-2.2.4.1/keras/engine/training_utils.py", line 143, in standardize_input_data
    str(data_shape))
ValueError: Error when checking input: expected input_1 to have shape (None, None, 3) but got array with shape (3, 299, 299)

Can you tell me do I need to modify anything? By the way, I am using the source code and I use tensorflow as backend for keras instead of theano. Is it cause any problem?

Thanks a lot!

[MarcBS]

As you mention the problem is caused because you are using tensorflow instead of theano, and the order of the image dimensions is different by default. You should change it on keras' configuration in ~/.keras/keras.json.

Just change the image_dim_ordering parameter to th instead of tf.

[bury-source]

@MarcBS

Thanks a lot! It solved the problem.

[bury-source]

Hi, I start training the Ingredients model and when it's going to evaluate model outputs on val set, it throws out an error:

Epoch 1/200
3400/3409 [============================>.] - ETA: 0s - loss: 0.0626[07/01/2019 22:03:51] <<< Predicting outputs of val set >>>
[07/01/2019 22:04:05] Evaluating on metric multilabel_metrics
Traceback (most recent call last):
  File "main.py", line 199, in <module>
    train_model(params)
  File "main.py", line 73, in train_model
    ing_model.trainNet(dataset, training_params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 677, in trainN
et    self.__train(ds, params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 862, in __trai
n    initial_epoch=params['epoch_offset'])
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/engine/training.py", line 1565, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/callbacks.py", line 43, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 170, in 
on_epoch_end    self.evaluate(epoch, counter_name='epoch')
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 290, in 
evaluate    split=s)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/evaluation.py", line 146, in
 multilabel_metrics    y_pred[i_s, word2idx[word]] = 1
KeyError: 0.69028074

It seems that the argument pred_list of method multilabel_metrics in evaluation.py is composed of probability rather that word dictionary. Is there any clue to solve the problem? Thanks a lot!

BTW: I'm using custom keras==1.2.3, multimodal_keras_wrapper-0.7, with backend=tensorflow-gpu==1.5.0, python2.7, run train_recipe5k.sh

[bury-source]

Hi, I start training the Ingredients model and when it's going to evaluate model outputs on val set, it throws out an error:

Epoch 1/200
3400/3409 [============================>.] - ETA: 0s - loss: 0.0626[07/01/2019 22:03:51] <<< Predicting outputs of val set >>>
[07/01/2019 22:04:05] Evaluating on metric multilabel_metrics
Traceback (most recent call last):
  File "main.py", line 199, in <module>
    train_model(params)
  File "main.py", line 73, in train_model
    ing_model.trainNet(dataset, training_params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 677, in trainN
et    self.__train(ds, params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 862, in __trai
n    initial_epoch=params['epoch_offset'])
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/engine/training.py", line 1565, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/callbacks.py", line 43, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 170, in 
on_epoch_end    self.evaluate(epoch, counter_name='epoch')
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 290, in 
evaluate    split=s)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/evaluation.py", line 146, in
 multilabel_metrics    y_pred[i_s, word2idx[word]] = 1
KeyError: 0.69028074

It seems that the argument pred_list of method multilabel_metrics in evaluation.py is composed of probability rather that word dictionary. Is there any clue to solve the problem? Thanks a lot!

BTW: I'm using custom keras==1.2.3, multimodal_keras_wrapper-0.7, with backend=tensorflow-gpu==1.5.0, python2.7, run train_recipe5k.sh

I think I solved the problem by modifying multimodal_keras_wrapper-0.7\multimodal_keras_wrapper-0.7\keras_wrapper\extra\callbacks.py, line 254, change elif to else statement, because I found the self.is_multilabel is always False `

elif self.is_multilabel:

else: ` After modifying this file, I got another error:

Traceback (most recent call last):
  File "main.py", line 199, in <module>
    train_model(params)
  File "main.py", line 73, in train_model
    ing_model.trainNet(dataset, training_params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 677, in trainN
et    self.__train(ds, params)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/cnn_model.py", line 862, in __trai
n    initial_epoch=params['epoch_offset'])
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/engine/training.py", line 1565, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/disk/lucy/sourceCode/keras-1.2.3/keras-1.2.3/keras/callbacks.py", line 43, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 170, in
on_epoch_end    self.evaluate(epoch, counter_name='epoch')
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/callbacks.py", line 296, in
evaluate    split=s)
  File "/disk/lucy/sourceCode/multimodal_keras_wrapper-0.7/multimodal_keras_wrapper-0.7/keras_wrapper/extra/evaluation.py", line 154, in
multilabel_metrics    gt_list = extra_vars[split]['references']
KeyError: 'references'

I modified the main.py in this repository, line 162, adding the following code: exec("extra_vars[s]['references'] = dataset.Y_"+s+"[params['OUTPUTS_IDS_DATASET'][0]]")