Same prediction for all inputs with inception model

[kushia]

Hi everyone! I trained an inception model with my custom images (dataset of 8.000 images), then saved it. And when I try to make predictions with it, whatever is the input images to predict, it's the same output! I checked everything, and all was seeming good to me...

# Here is the code to train the model `import os import numpy as np import pandas as pd from scipy.misc import imread from scipy.misc import imresize from inception_v3 import InceptionV3 from keras.preprocessing import image from keras.models import Model from keras.layers import Dense, GlobalAveragePooling2D from keras import backend as K import keras from keras.callbacks import ModelCheckpoint root_dir = os.path.abspath('.') data_dir = os.path.join(root_dir, 'data') os.path.exists(root_dir) os.path.exists(data_dir) nb_epoch = int(raw_input("Nb epoch ? : ")) print ('..Debut parsing et resize..') train = pd.read_csv(os.path.join(data_dir, 'images', 'test.csv')) train.head() temp = [] for img_name in train.filename: image_path = os.path.join(data_dir, 'images', img_name) img_tmp = imread(image_path, mode='RGB') img = imresize(img_tmp, (299, 299)) img = img.astype('float32') temp.append(img) print ('..Fin parsing et resize..') train_x = np.stack(temp) train_x = train_x#, train_x[split_size:] train_y = keras.utils.np_utils.to_categorical(train.label.values) base_model = InceptionV3(weights=None, include_top=False)

x = base_model.output x = GlobalAveragePooling2D()(x) x = Dense(1024, activation='relu')(x) predictions = Dense(196, activation='softmax')(x)

model = Model(input=base_model.input, output=predictions)

print ('..Debut compilation du model..') model.compile(optimizer='rmsprop', loss='categorical_crossentropy') print ('..Fin compilation du model..') filepath="weights-improvement-{epoch:02d}.hdf5" checkpoint = ModelCheckpoint(filepath, verbose=1, save_best_only=True, mode='max') callbacks_list = [checkpoint] print ('Debut entrainement') trained_model = model.fit(train_x, train_y, nb_epoch=nb_epoch, callbacks=callbacks_list, batch_size=8) model.save('car_model.h5') print ('Fin de l\'entrainement') `

## And here is the code to make predictions

`import os import numpy as np import pandas as pd from scipy.misc import imread from scipy.misc import imresize from inception_v3 import InceptionV3 from inception_v3 import preprocess_input from keras.preprocessing import image from keras.models import Model from keras.layers import Dense, GlobalAveragePooling2D from keras import backend as K import keras from keras.callbacks import ModelCheckpoint from imagenet_utils import decode_predictions root_dir = os.path.abspath('.') data_dir = os.path.join(root_dir, 'data') os.path.exists(root_dir) os.path.exists(data_dir)

base_model = InceptionV3(weights=None, include_top=False)

x = base_model.output x = GlobalAveragePooling2D()(x) x = Dense(1024, activation='relu')(x) predictions = Dense(197, activation='softmax')(x)

model = Model(input=base_model.input, output=predictions)

print ('..Debut compilation du model..') model.load_weights('car_model.h5') model.compile(optimizer='rmsprop', loss='categorical_crossentropy') print ('..Fin compilation du model..')

while True:

#img_path = raw_input('Img path : ')

img_path = '/home/images/truck1.jpg' # Pas propre. A changer print (img_path) img_tmp = imread(img_path, mode='RGB') img = imresize(imgtmp, (299, 299)) img = img.astype('float32') x = np.expanddims(img, axis=0) preds = model.predict(x) decode_predictions(preds) print ('Fin prediction') `

I hope you'll find the problem, because I don't see at all..

(I trained the model over 50 epochs, so the problem don't come for here)

[toqitahamid]

What are you doing in this method decode_predictions(preds)? Can you share the code?

[kushia]

Hi, sure : def decode_predictions(preds, top=5): results = [] for pred in preds: top_indices = pred.argsort()[-top:][::-1] for i in top_indices : print ("prediction is " + i) return "Prediction Over"

This function is inspired by the function "decode_pred" in the file imagenet_utils.py But it always return the same top-5 labels no matter what's the input image I try..

[leminhlong-pixta]

I have the same problem. Logits are the same even though inputs are different for inception v4. I also use my custom code for predicting tags for images.

import tensorflow as tf
import inception_base_model
import math
from datetime import datetime
from image_processing import ImageProcessing
import numpy as np
from collections import OrderedDict
from lib.inception_model import inception_base_model as inception

class InceptionOutput(object):
  def __init__(self, checkpoint_dir):
    self.checkpoint_dir = checkpoint_dir

  def output(self, image, num_classes, vocab, threshold=0.5):
    with tf.Session() as sess:
      img_processing = ImageProcessing()
      image = img_processing.process_image(image)
      logits, _ = inception_base_model.inference(image, num_classes=num_classes,
                                                 for_training=False)
      ckpt = tf.train.get_checkpoint_state(self.checkpoint_dir)
      if ckpt:
        variable_averages = tf.train.ExponentialMovingAverage(
          inception.MOVING_AVERAGE_DECAY)
        variables_to_restore = variable_averages.variables_to_restore()
        restorer = tf.train.Saver(variables_to_restore)
        checkpoint_path = ckpt.model_checkpoint_path
        restorer.restore(sess, checkpoint_path)
        print('%s: Pre-trained model restored from %s' %
              (str(datetime.now()), str(checkpoint_path)))

        # Assuming model_checkpoint_path looks something like:
        #   /my-favorite-path/imagenet_train/model.ckpt-0,
        # extract global_step from it.
        global_step = ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1]
        print('Succesfully loaded model from %s at step=%s.' %
              (ckpt.model_checkpoint_path, global_step))
      else:
        print('No checkpoint file found')
        sess.run(tf.initialize_all_variables())
      logits = sess.run(logits)
      return self.get_top_dict(logits[0], vocab, threshold)

  def get_top_dict(self, output_logits, rev_target_vocab, threshold):
    trans_confidence_dict = OrderedDict()
    temp = np.argsort(output_logits)
    top_logit_indices = temp[-5:]
    for logit_index in top_logit_indices:
      # Numpy array has one element. Inside that element is a list of logits for vocab
      trans_logit = output_logits[logit_index]
      #Continue with if the logit index does not exist
      if len(rev_target_vocab) <= logit_index:
        continue
      trans = tf.compat.as_str(rev_target_vocab[logit_index])
      # Faster than tensorflow's sigmoid.
      confidence = 1.0 / (1.0 + math.exp(-trans_logit))
      if (trans not in trans_confidence_dict and confidence >= threshold) or \
        (trans in trans_confidence_dict and confidence > trans_confidence_dict[trans]):
        # Add confidence and translation to dictionary if the key has higher confidence or
        # if the key doesn't exist in dictionary.
        trans_confidence_dict[trans] = confidence
    return trans_confidence_dict

[gugarosa]

I'm also having the same problem as you! No matter which network I use, I always get the same predictions for different images. This only happens when I try to train my own network and save either its whole configuration or weights.

If I'm using the pre-built weights from VGG, ResNet, whatever, it seems to be working fine.

[oleksandr-kovalov]

I have exactly the same problem for keras 2.0.3(tensorflow) and InceptionV3 (custom trained) model. Pre-build weights from VGG, InceptionV3 works just fine. But if I train my own top layers - model gives the same prediction for any object.

My model has val_acc is ~95-97% according to output from training and validation using fit_generator

[kushia]

Did you forget to preprocess your input for the prediction ?

[leminhlong-pixta]

Preprocessing solved my problem

[triployd]

helllllllllo, @leminhlong-pixta i have the same problem as you described above. can you tell me a bit more specific about "preprocessing" ? did you solve the problem by subtracting mean or something ? thanks!

[PythonImageDeveloper]

@leminhlong-pixta , Hi , i've trained inception-v4 for my own dataset but i don;t know for prediction for test images , have you some code about prediction ? i use tf-slim library.

[ividal]

Having the same problem: output class is always the same for all images I input, even if I do predictions one by one. The next time I load the model, the "favourite" class changes, but still applies to all input images.

I'm using ImageDataGenerator with the corresponding preprocess_input function, then flow_from_directory to feed the images.

Observed in: Tensorflow's Keras 2.1.4-tf with Python3.6 on Ubuntu 16.04.

• Keras pretrained VGG16, using vgg16 preprocess_input inside my ImageDataGenerator. Loading with model = VGG16(weights="imagenet")
• Keras pretrained InceptionV3, using inception_v3 preprocess_input, loading with model = InceptionV3(weights="imagenet")

I load my data with:

    datagen = ImageDataGenerator(preprocessing_function=preprocess_input)

    datagen.flow_from_directory(
            data_root_dir, target_size=(image_size[0:2]),
            batch_size=1, save_to_dir='preview', save_prefix='t',
            save_format='jpeg')

And then do the predictions with:

labels_out = model.predict(image)

I see the same behaviour with model.predict_on_batch(image)

Any ideas...?

[R-Miner]

I have the same problem. I did the preorpocessing of the input image as well. Anyone solved the issue?

[tom-neumark]

I have the same issue. I ran retrain.py against the flowers example. I ran label_image.py and it worked as expected. Then I tried to load the model using the LabelImage.java example (passing in Placeholder as input and final_results as output) and I always get the same prediction.

[DongChen06]

Anybody solves the problem? Thank you!

[geemmm]

After img = img.astype('float32'), try to add img = img/255. to make sure every value is between 0 and 1.

[slimguat]

After img = img.astype('float32'), try to add img = img/255. to make sure every value is between 0 and 1.

this was my problem thanks brah

[zeshankhan]

I got the same issue the increase in the training dataset and epochs makes the output probabilities similar for different images. At <10 epochs and dataset of size 10 the model is predicting different probabilities for different images but for more than 100 epochs and dataset of 1000s it generates an exactly same probability distribution for each class.

[Shivam1603]

I am having the same issue. Has anyone solved it? I have tried pre-processing, still having the same trouble.