Getting gray image when I try to run prediction

[Linardos]

I'm trying to test if I can run this model correctly. I have gone through the preprocess step and tried to train for 3 epochs just to see that it runs; however, when I load the model state to check what it predicts I get completely gray images. Obviously 3 epochs is too low for training, but is it normal to get completely homogeneous gray images?

import torch
import torch.nn as nn
from utils import predict
import cv2
from generator import Generator
from constants import *
import pickle
import numpy as np

validation_sample = cv2.imread("images256x192_val/COCO_val2014_000000285005.png")

validation_truth = cv2.imread("maps256x192_val/COCO_val2014_000000285005.png")

cv2.imshow('sample',np.hstack([validation_sample, validation_truth]))
cv2.waitKey()
#The image and the ground truth is correctly displayed

generator = Generator()
generator.load_state_dict(torch.load("./generator.pkl", map_location=lambda storage, loc: storage))

print(type(generator))
predict(generator, validation_sample, 1, "./test/")
img = cv2.imread('./test/1.png')
cv2.imshow('test',img)
cv2.waitKey()
#All I am getting is a completely gray image

[batsa003]

Can you refer to Predict.ipynb for running a pre-trained model? If it is still gray image, you may want to train more.

[Linardos]

I am now using the pre-trained weights from the original model, ported on your generator; however, I am still getting a completely gray image. I follow the instructions from Predict.ipynb to the book but it seems that the output of the model is just a homogeneous matrix with 0.4 values everywhere, hence the blank image. Are you maybe using a different preprocess than the original authors did? I also noticed that the shape of some of your layers differs to the original one, their 1st and 2nd dimensions switched (38,44,48 and 52 layers) for example:

Layer 48: Pytorch implementation: torch.Size([128, 64, 3, 3]) Lasagne implementation: torch.Size([64, 128, 3, 3])

In order to successfully port the weights, in spite of this, I reshaped the original weights to fit into your model. But maybe the root of this discrepancy is the cause of the blank result?

[tr717]

I have the same problem. Have you solved it now? thanks

[Linardos]

I have the same problem. Have you solved it now? thanks

I never solved it with this implementation. A colleague of mine ended up making another implementation on PyTorch; you may use that one: https://github.com/imatge-upc/saliency-2019-SalBCE

[tr717]

Thank you

[DilaraAlbayrak]

@Linardos is the implementation for original salgan or does it use only BCE loss? By the way, I have similar problem with some other implementation, which I have just referred here, fyi.