Open gaody9527 opened 6 years ago
My solution was not to define the model as sequential, but to add the target layer directly.
x = Lambda(target_layer, output_shape = target_category_loss_output_shape)(input_model.output)
model = Model(inputs=input_model.input, outputs=x)
The grad_cam function looks like this now:
nb_classes = 1000
target_layer = lambda x: target_category_loss(x, category_index, nb_classes)
x = Lambda(target_layer, output_shape = target_category_loss_output_shape)(input_model.output)
model = Model(inputs=input_model.input, outputs=x)
model.summary()
loss = K.sum(model.output)
conv_output = [l for l in model.layers if l.name is layer_name][0].output
grads = normalize(_compute_gradients(loss, [conv_output])[0])
gradient_function = K.function([model.input], [conv_output, grads])
output, grads_val = gradient_function([image])
output, grads_val = output[0, :], grads_val[0, :, :, :]
weights = np.mean(grads_val, axis = (0, 1))
cam = np.ones(output.shape[0 : 2], dtype = np.float32)
for i, w in enumerate(weights):
cam += w * output[:, :, i]
cam = cv2.resize(cam, (224, 224))
cam = np.maximum(cam, 0)
heatmap = cam / np.max(cam)
#Return to BGR [0..255] from the preprocessed image
image = image[0, :]
image -= np.min(image)
image = np.minimum(image, 255)
cam = cv2.applyColorMap(np.uint8(255*heatmap), cv2.COLORMAP_JET)
cam = np.float32(cam) + np.float32(image)
cam = 255 * cam / np.max(cam)
return np.uint8(cam), heatmap´´´
@kobotschick Thanks you for your proposals, I have dealed with the problem to follow the https://github.com/vense/keras-grad-cam/blob/master/grad-cam.py
@kobotschick One problem I had with your solution was a message that a list doesn't have an attributed called dtype. The solution was to change gradient_function = K.function([model.input], [conv_output, grads])
to gradient_function = K.function(model.input, [conv_output, grads])
@kobotschick Where is Model() function defined?
full grad_cam function along with import as follows:
from keras.models import Model
def grad_cam(input_model, image, category_index, layer_name):
nb_classes = 1000
target_layer = lambda x: target_category_loss(x, category_index, nb_classes)
x = Lambda(target_layer, output_shape = target_category_loss_output_shape)(input_model.output)
model = Model(inputs=input_model.input, outputs=x)
model.summary()
loss = K.sum(model.output)
conv_output = [l for l in model.layers if l.name is layer_name][0].output
grads = normalize(_compute_gradients(loss, [conv_output])[0])
gradient_function = K.function([model.input], [conv_output, grads])
output, grads_val = gradient_function([image])
output, grads_val = output[0, :], grads_val[0, :, :, :]
weights = np.mean(grads_val, axis = (0, 1))
cam = np.ones(output.shape[0 : 2], dtype = np.float32)
for i, w in enumerate(weights):
cam += w * output[:, :, i]
cam = cv2.resize(cam, (224, 224))
cam = np.maximum(cam, 0)
heatmap = cam / np.max(cam)
#Return to BGR [0..255] from the preprocessed image
image = image[0, :]
image -= np.min(image)
image = np.minimum(image, 255)
cam = cv2.applyColorMap(np.uint8(255*heatmap), cv2.COLORMAP_JET)
cam = np.float32(cam) + np.float32(image)
cam = 255 * cam / np.max(cam)
return np.uint8(cam), heatmap
For everyone who is facing the same issue: I forked this repository and fixed all errors I was facing.
https://github.com/jacobgil/keras-grad-cam/pull/27 https://github.com/PowerOfCreation/keras-grad-cam
The following change works for me :
gradient_function = K.function([model.layers[0].get_input_at(1)], [conv_output, grads])
instead of
gradient_function = K.function([model.layers[0].get_input_at(0)], [conv_output, grads])
when I run the code ,the following question appeared AttributeError: Layer vgg16 has multiple inbound nodes, hence the notion of "layer input" is ill-defined. Use
get_input_at(node_index)
instead. i change it as gradient_function = K.function([model.layers[0].get_input_at(1)], [conv_output, grads]) but it also does not work i do not understand what the function want? any help will be appreciated.