keisen
commented
4 years ago Hi, @MarcoLengua . Thank you for your report.
Unfortunately, keras-vis can't generate the saliency map of your model. The direct causes of problem seem that keras-vis can't find target layer that is contained cascading model. But even if keras-vis got the layer using visualize_saliency_with_losses instead of visualize_saliency, you find out another fundamental problem that keras can't calculate gradients for nested models. That's, your scenario is same as the 1st case of following page:
I have an issue with visualizing my conv layers in the following model architecture, because keras-vis seems not to be able to handle model in models architecture. I want to visualize a conv layer within my vgg19 transfered model. But i can't acces them. Does anyone know how to visualize the conv layers in a model within a model architecture or even how to "flatten out" the vgg19 model so i can access all layers. It seems to be an open issue of keras-vis and i need the visualization urgently for my bachelor thesis. Any help or recommendation of another visualization tool is greatly appreciated! Thank you!
Layer (type) Output Shape Param #
input_1 (InputLayer) (None, 120, 160, 1) 0
lambda_1 (Lambda) (None, 120, 160, 3) 0
batch_normalization_1 (Batch (None, 120, 160, 3) 12
vgg19 (Model) multiple 20024384
flatten_1 (Flatten) (None, 7680) 0
dense_1 (Dense) (None, 80) 614480
dropout_1 (Dropout) (None, 80) 0
dense_2 (Dense) (None, 4800) 388800
reshape_1 (Reshape) (None, 60, 80) 0
Total params: 21,027,676 Trainable params: 1,003,286 Non-trainable params: 20,024,390
None
The VGG19 model consists of the following layers, that i can't access!
['input_2', 'block1_conv1', 'block1_conv2', 'block1_pool', 'block2_conv1', 'block2_conv2', 'block2_pool', 'block3_conv1', 'block3_conv2', 'block3_conv3', 'block3_conv4', 'block3_pool', 'block4_conv1', 'block4_conv2', 'block4_conv3', 'block4_conv4', 'block4_pool', 'block5_conv1', 'block5_conv2', 'block5_conv3', 'block5_conv4', 'block5_pool']
model = load_model('/home/marco/HyperOptBachelor/vgg19_final100epochs/weights.hdf5', custom_objects={'motion_metric': motion_metric}) print(model.summary()) print([layer.name for layer in model.get_layer('vgg19').layers]) layer_idx = utils.find_layer_idx(model, 'block5_conv4')
This is just a teacher frame from my test dataset, but it can to test it any image
img = utils.load_img('/home/marco/HyperOptBachelor/vgg19_final100epochs/out/test-96-teacher.png') f, ax = plt.subplots(1,1) grads = visualize_saliency(model,-5,filter_indices=10,seed_input=img) ax[1].imshow(grads, cmap='jet')
When i try to get the index of a specific layer in that vgg19 model it comes up with the following error.
Traceback (most recent call last): File "/home/marco/PycharmProjects/ideal-system/helper_scripts/visualization.py", line 11, in
layer_idx = utils.find_layer_idx(model, 'block5_conv4')
File "/home/marco/bachelor/lib/python3.5/site-packages/vis/utils/utils.py", line 164, in find_layer_idx
raise ValueError("No layer with name '{}' within the model".format(layer_name))
ValueError: No layer with name 'block5_conv4' within the model
so it doesnt find that layer because theres just the model not all the layers listed i guess. And if i just call by index the vgg19 model it throws an error about that the model has mutliple output nodes. I searched the internet and it seems to be an open issue. but now i dont know how to keep progressing!
Kind Regards,
Marco Lengua