Inaccurate keras model output after converting a finetuned Resnet50 Pyotrch model.

[SKRohit]

I had finetuned a Resnet50 for the binary classification task in PyTorch. It is performing quite well. But the converted Keras model is always choosing only one class (the class with index value 1) for images belonging to any class. I used just transforms.Resize((224,224)), transforms.CenterCrop(224), transforms.ToTensor() transforms in PyTorch while in Keras I am only resizing the images to (224,224).

# defining PyTorch model
severity_model = models.resnet50(pretrained=True)
severity_model.fc = nn.Sequential(
    nn.Linear(2048, 1024),
    nn.ReLU(inplace=True),
    nn.Dropout(),
    nn.Linear(1024, 512),
    nn.ReLU(inplace=True),
    nn.Dropout(),
    nn.Linear(512, 256),
    nn.ReLU(inplace=True),
    nn.Dropout(),
    nn.Linear(256, 128),
    nn.ReLU(inplace=True),
    nn.Dropout(),
    nn.Linear(128, 2),
)
# Load weights
device = torch.device("cpu")
severity_model.load_state_dict(torch.load("./Model_Weights/best_resnet50_large_new_data_300epoch", map_location=device))

from pytorch2keras import pytorch_to_keras
# we should specify shape of the input tensor
input_np = np.zeros((1,3, 224, 224),dtype=np.float32)
k_model = pytorch_to_keras(severity_model, input_var, [(3, None, None,)], verbose=True, change_ordering=True)

# saving keras model
k_model.save('my_model.h5')

# loading keras model
new_model = tf.keras.models.load_model('my_model.h5')

# prediction using keras model
new_model.predict(np.asarray(img, dtype=np.float32)[np.newaxis, :, :, :])

Any suggestion on what might be going wrong in Keras model? And how to resolve it?

Environment:

• OS: [Win10]
• Python [Python 3.6.10]
• pyotrch2keras [0.2.3]
• keras [2.2.4]
• tensorflow [1.5.0]
• pytorch [1.4.0] [cpuonly]

[gmalivenko]

Probably you forgot about .eval() call for your model. I'd recommend you to compare outputs for the same input (e.g., fixed random tensor) both for PyTorch and converted Keras model.

[SKRohit]

@nerox8664 yes I am verifying it on the same input and I am running the input in eval mode for PyTorch model.

[SKRohit]

@nerox8664 I resolved it. I was not scaling the input image i.e. my input image was in the range [0,255] while feeding it to the model. But after dividing the image by 255 the converted model gives me proper classification output.