Keras-Vis for custom layer

[tonystark12]

First off, thank you for providing,contributing and making this package, which has been helpful to me for many research based projects.

I am facing the following error statement

    ValueError: Unknown layer: AcceptRejectPooling2D

when trying to run keras-vis. The code block causing the error :

    layer_idx = utils.find_layer_idx(model, 'dense_2') #after naming the layers
    model.layers[layer_idx].activation = activations.softmax
    model = utils.apply_modifications(model)

for class_idx in np.arange(10):    
     indices = np.where(y_train[:, class_idx] == 1.)[0]
     idx = indices[0]

     grads = visualize_saliency(model, layer_idx, filter_indices=class_idx, 
                           seed_input=X_train[idx], 
     backprop_modifier='guided')

     f, ax = plt.subplots(1, 2)
     ax[0].imshow(X_train[idx][..., 0])
     ax[1].imshow(grads, cmap='jet')

     savename = "layer_activations" + str(class_idx) + ".png"
     f.savefig(savename)

I have used keras-vis for inbuilt layers and it works great. How do I make use of it for custom layers? I am using WIN 10, with python 3.6.5, and jupyter 4.4.0

Thanks in advance. Please let me know if any other info can help.

[tonystark12]

Edit : The complete error :

---

ValueError Traceback (most recent call last)

in () 1 layer_idx = utils.find_layer_idx(model, 'dense_3') 2 model.layers[layer_idx].activation = activations.softmax ----> 3 model = utils.apply_modifications(model) 4 5 for class_idx in np.arange(10): c:\python36\lib\site-packages\vis\utils\utils.py in apply_modifications(model, custom_objects) 111 try: 112 model.save(model_path) --> 113 return load_model(model_path, custom_objects=custom_objects) 114 finally: 115 os.remove(model_path) c:\python36\lib\site-packages\keras\engine\saving.py in load_model(filepath, custom_objects, compile) 417 f = h5dict(filepath, 'r') 418 try: --> 419 model = _deserialize_model(f, custom_objects, compile) 420 finally: 421 if opened_new_file: c:\python36\lib\site-packages\keras\engine\saving.py in _deserialize_model(f, custom_objects, compile) 223 raise ValueError('No model found in config.') 224 model_config = json.loads(model_config.decode('utf-8')) --> 225 model = model_from_config(model_config, custom_objects=custom_objects) 226 model_weights_group = f['model_weights'] 227 c:\python36\lib\site-packages\keras\engine\saving.py in model_from_config(config, custom_objects) 456 '`Sequential.from_config(config)`?') 457 from ..layers import deserialize --> 458 return deserialize(config, custom_objects=custom_objects) 459 460 c:\python36\lib\site-packages\keras\layers\__init__.py in deserialize(config, custom_objects) 53 module_objects=globs, 54 custom_objects=custom_objects, ---> 55 printable_module_name='layer') c:\python36\lib\site-packages\keras\utils\generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name) 143 config['config'], 144 custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) + --> 145 list(custom_objects.items()))) 146 with CustomObjectScope(custom_objects): 147 return cls.from_config(config['config']) c:\python36\lib\site-packages\keras\engine\sequential.py in from_config(cls, config, custom_objects) 298 for conf in layer_configs: 299 layer = layer_module.deserialize(conf, --> 300 custom_objects=custom_objects) 301 model.add(layer) 302 if not model.inputs and build_input_shape: c:\python36\lib\site-packages\keras\layers\__init__.py in deserialize(config, custom_objects) 53 module_objects=globs, 54 custom_objects=custom_objects, ---> 55 printable_module_name='layer') c:\python36\lib\site-packages\keras\utils\generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name) 136 if cls is None: 137 raise ValueError('Unknown ' + printable_module_name + --> 138 ': ' + class_name) 139 if hasattr(cls, 'from_config'): 140 custom_objects = custom_objects or {} ValueError: Unknown layer: AcceptRejectPooling2D IS the error related to https://github.com/keras-team/keras/pull/5995 ?

[keisen]

Hi, @tonystark12 . Your guessed is right. Could you try following? :

from keras.utils import CustomObjectScope
with CustomObjectScope({'AcceptRejectPooling2D': AcceptRejectPooling2D}): 
    model = utils.apply_modifications(model)

and

with CustomObjectScope({'AcceptRejectPooling2D': AcceptRejectPooling2D}): 
     grads = visualize_saliency(model, layer_idx, filter_indices=class_idx, seed_input=X_train[idx], backprop_modifier='guided')

[tonystark12]

hey @keisen

Thank you for your reply. This is the error I got now. based on the changes I made.

TypeError: ('Keyword argument not understood:', 'strides')

I feel the custom ayer is not being recognized. Is there some form of initialization that I am missing? The complete code is attached along with this comment, in case required.

modifiedstochastic.py - the custom layer defined codes.zip

[tonystark12]

The complete error o/p here :

TypeError Traceback (most recent call last)

in () 2 model.layers[layer_idx].activation = activations.softmax 3 with CustomObjectScope({'AcceptRejectPooling2D': AcceptRejectPooling2D}): ----> 4 model = utils.apply_modifications(model) 5 6 for class_idx in np.arange(10): c:\python36\lib\site-packages\vis\utils\utils.py in apply_modifications(model, custom_objects) 111 try: 112 model.save(model_path) --> 113 return load_model(model_path, custom_objects=custom_objects) 114 finally: 115 os.remove(model_path) c:\python36\lib\site-packages\keras\engine\saving.py in load_model(filepath, custom_objects, compile) 417 f = h5dict(filepath, 'r') 418 try: --> 419 model = _deserialize_model(f, custom_objects, compile) 420 finally: 421 if opened_new_file: c:\python36\lib\site-packages\keras\engine\saving.py in _deserialize_model(f, custom_objects, compile) 223 raise ValueError('No model found in config.') 224 model_config = json.loads(model_config.decode('utf-8')) --> 225 model = model_from_config(model_config, custom_objects=custom_objects) 226 model_weights_group = f['model_weights'] 227 c:\python36\lib\site-packages\keras\engine\saving.py in model_from_config(config, custom_objects) 456 '`Sequential.from_config(config)`?') 457 from ..layers import deserialize --> 458 return deserialize(config, custom_objects=custom_objects) 459 460 c:\python36\lib\site-packages\keras\layers\__init__.py in deserialize(config, custom_objects) 53 module_objects=globs, 54 custom_objects=custom_objects, ---> 55 printable_module_name='layer') c:\python36\lib\site-packages\keras\utils\generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name) 143 config['config'], 144 custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) + --> 145 list(custom_objects.items()))) 146 with CustomObjectScope(custom_objects): 147 return cls.from_config(config['config']) c:\python36\lib\site-packages\keras\engine\sequential.py in from_config(cls, config, custom_objects) 298 for conf in layer_configs: 299 layer = layer_module.deserialize(conf, --> 300 custom_objects=custom_objects) 301 model.add(layer) 302 if not model.inputs and build_input_shape: c:\python36\lib\site-packages\keras\layers\__init__.py in deserialize(config, custom_objects) 53 module_objects=globs, 54 custom_objects=custom_objects, ---> 55 printable_module_name='layer') c:\python36\lib\site-packages\keras\utils\generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name) 145 list(custom_objects.items()))) 146 with CustomObjectScope(custom_objects): --> 147 return cls.from_config(config['config']) 148 else: 149 # Then `cls` may be a function returning a class. c:\python36\lib\site-packages\keras\engine\base_layer.py in from_config(cls, config) 1107 A layer instance. 1108 """ -> 1109 return cls(**config) 1110 1111 def count_params(self): D:\Freelancing\CNN_analysis\code\modifiedstochastic.py in __init__(self, pool_size, padding, data_format, **kwargs) 17 class AcceptRejectPooling2D(Layer): 18 def __init__(self, pool_size=(2, 2), padding='SAME', data_format='channels_last', **kwargs): ---> 19 super(AcceptRejectPooling2D, self).__init__(**kwargs) 20 21 self.pool_size = conv_utils.normalize_tuple(pool_size, 2, 'pool_size') c:\python36\lib\site-packages\keras\engine\base_layer.py in __init__(self, **kwargs) 126 for kwarg in kwargs: 127 if kwarg not in allowed_kwargs: --> 128 raise TypeError('Keyword argument not understood:', kwarg) 129 name = kwargs.get('name') 130 if not name: TypeError: ('Keyword argument not understood:', 'strides')

[keisen]

Can you modify modifiedstochastic.py to that comment out the 159 lines? Or add strides argument to __init__ method?

[tonystark12]

@keisen

Thank you for your patience again. Could you please elaborate more on adding strides to init.py? I have created an empty file for project purposes, and thus do not have any functions there.

Doesn't commenting out strides affect the layer data ?

[tonystark12]

Commenting out strides helped. Thank you once again for your time. If possible. could you please give me a gist or explaining the issue I faced?

[keisen]

When a model is serialized, the config in dict of return value of Layer#get_config() is saved as one of model's parameters. Then when the model is deserialized, the config value is passed to __init__() method as arguments.

The error you faced was occured becase deserialized strides value can't be passed the __init__() as arguments.

....Maybe.

I am not familiar with keras's design, so I might make mistake.