mangotee
commented
4 years ago Hi there,
sorry for the late response. I've taken a look at your problem today. Initially I thought that the highly colored image might be the problem - be aware that we trained this for neuroscientific applications, most of which operate on grey-valued videos, and the colored images we used were still very low in saturation. However, that is not the issue here (grey-valued images do improve the prediction though, see below).
Your code has an issue in this line:
reader.resize((240,320))
First of all, this does not run for me, python throws an error. What you are trying to achieve here is a resizing of the image from its original dimensions (374x709) to the required input dimensions (240x320). You're using the numpy function resize for this, which does not work (for me), at least not as expected (for you). The resize operation gives a return value which you're not catching. Also, after a bit of tweaking, the output is a black image (and thus, the prediction is also black). The function you're looking for is skimage.transform.resize, which not only resizes to the desired array dimensions, but also resamples/interpolates the image properly. After that, the prediction works as expected, actually much better than I thought it would (given the high saturation of the image).
Apart from that, I also tried the conversion of color to grayscale, because I expect this to work better, and it does, the prediction is a bit sharper and the tear-sac yields a lower false-positive confidence of detection (i.e. it's dark in both images, but on grey-valued images, the model seemingly learned better to reject such dark portions, given it's location w.r.t. the surrounding anatomy).
Here's the code you can run to confirm that it's actually working on this image. My output is attached.
from DeepVOG_model import load_DeepVOG
import skimage.io as ski
import skimage.transform as skt
import skimage.color as skc
import numpy as np
from matplotlib import pylab as plt
# load model and image
model = load_DeepVOG()
img = np.zeros((1, 240, 320, 3))
reader = ski.imread("test_image_rgb_issue6.png")/255
# Pre-processing to different versions
# RGB version
img_scaled = skt.resize(reader,(240,320))#
img_rgb = img_scaled.reshape(1, 240, 320, 3)
# Gray-valued version
img_scaled_gray = skc.gray2rgb( skc.rgb2gray(img_scaled) )
img_gray = img_scaled_gray.reshape(1, 240, 320, 3)
# Prediction
prediction_rgb = model.predict(img_rgb)
prediction_gray = model.predict(img_gray)
# visualization of input and output side-by-side
fig, axes = plt.subplots(2, 2, figsize=(15,10))
axes[0,0].imshow(np.squeeze(img_rgb))
axes[0,1].imshow(np.squeeze(img_gray))
axes[1,0].imshow(np.squeeze(prediction_rgb))
axes[1,1].imshow(np.squeeze(prediction_gray))
bmionescu
I tried running the "test_if_model_work.py" file. The test_image.png file included with the python and h5 files didn't work, so I just took an image off google images. However, the output image is black. The numbers in the outputted array are on the order of 10^-5, 10^-6 and 10^-7. When I tried scaling these to the 0-255 range, the output was just grayscale noise. I've included the image for completeness. I only slightly modified test_if_model_works, and did not modify anything else. Here is the slightly modified test_if_model_work code I used:
(Edit: I downloaded all of the codes in the DeepVOG folder, but it is not clear which ones are dependencies and which is the main code that runs everything.)
def test_if_model_work(): model = load_DeepVOG() img = np.zeros((1, 240, 320, 3)) reader=ski.imread("test_image.png")/255 reader.resize((240,320)) img[:,:,:,:] = reader.reshape(1, 240, 320, 1) prediction = model.predict(img) ski.imsave("test_prediction.png", prediction[0,:,:,1])
print(prediction)