Closed JaspervDalen closed 5 years ago
Hi, we used Lucid as well (we mentioned and linked to Lucid in the article). We haven't open sourced the code for the filter visualizations yet, will try to do that over the Christmas period
Here is a minimal example to visualize the first filter in layer 23 in MobileNet, it should produce
Hope it helps..
import os
import tensorflow as tf
import lucid.optvis.param as param
import lucid.optvis.render as render
import lucid.optvis.transform as transform
from lucid.modelzoo.vision_base import Model
from keras import backend as K
from keras.applications.mobilenet import MobileNet
import matplotlib.pyplot as plt
WORK_DIR = os.getcwd()
CONV_LAYER = 23
CONV_FILTER = 0
# save MobileNet in TF pb format
tf.reset_default_graph()
sess = tf.Session()
K.set_session(sess)
model_keras = MobileNet(include_top=False)
frozen_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
sess.graph_def,
[model_keras.outputs[-1].op.name],
)
with open(os.path.join(WORK_DIR, 'model_tf.pb'), 'wb') as f:
f.write(frozen_graph_def.SerializeToString())
# convert model for lucid
class Mobilenet_lucid(Model):
def __init__(self, model_path, labels_path, input_name, image_shape=None, image_value_range=None):
self.model_path = model_path
self.labels_path = labels_path
self.input_name = input_name
self.image_shape = image_shape
self.image_value_range = image_value_range
super().__init__()
# locations
image_shape = [224, 224, 3]
image_value_range = [-1, 1]
param_f = lambda: param.image(128, fft=True, decorrelate=True)
input_name = model_keras.input.name
model_path = os.path.join(WORK_DIR, 'model_tf.pb')
model_lucid = Mobilenet_lucid(model_path, None, input_name, image_shape, image_value_range)
model_lucid.load_graphdef()
layer_render = model_keras.layers[CONV_LAYER].output.name.split(':')[0]+':'+str(CONV_FILTER)
vis = render.render_vis(model_lucid, layer_render, param_f)
fig = plt.figure()
plt.imshow(vis[0][0][0, ])
plt.axis('off')
fig.savefig(os.path.join(WORK_DIR, 'vis_layer{}_filter{}.png'.format(CONV_LAYER, CONV_FILTER)))
Thank you for this example and for sharing your whole repo!
in this nvidia article (linked below) about your work you show layer visualizations of mobilenet. I tried to do that in a project of mine but it failed with multiple libraries (e.g. lucid : https://github.com/tensorflow/lucid/issues/68). So I was wondering how you did it.
article: https://devblogs.nvidia.com/deep-learning-hotel-aesthetics-photos/?nvid=em-ded-63644&mkt_tok=eyJpIjoiT1dGaE5ERm1NbVV5TmpreSIsInQiOiJOK0pzQnBxclNwUWNKNmNoNUNYbCtDVUlibzAzM21uYnNsRTBvZFhnWlk0RnJwZmltZHdRTFZTXC9Idk5vQXg2K3hjWE94cE9UU2NnV0xWVk1tWnhHSGhLUVZMbW84R1Z0QUN3RVNVaWdqS05yR1ZaR0FEbmFCMGlmNTZ5bndMNGcifQ%3D%3D