MobilenetV3 for Model Zoo

[tul-urte]

Will there be a MobilenetV3 model for the zoo?

Can Lucid handle "squeeze_excite/AvgPool"

I've tried using a basic MobilenetV3 Lucid model myself (like the MobilenetV2 example) but when I run the following code:

with tf.Graph().as_default(), tf.Session() as sess:
    T = render.make_vis_T( model, objective_f, param_f, optimizer, transforms )
    loss, vis_op, t_image = T("loss"), T("vis_op"), T("input")
    tf.global_variables_initializer().run() 
    threshold_loss, _ = sess.run( [ loss, vis_op ] )

I get the following error:

  File "../lib\vis.py", line 228, in get_visualizations_and_losses
    threshold_loss, _ = sess.run( [ loss, vis_op ] )
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\client\session.py", line 950, in run
    run_metadata_ptr)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\client\session.py", line 1173, in _run
    feed_dict_tensor, options, run_metadata)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_run
    run_metadata)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: Computed output size would be negative: -34 [input_size: 37, effective_filter_size: 72, stride: 1]
         [[node import/FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/AvgPool (defined at D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\modelzoo\vision_base.py:200) ]]
  (1) Invalid argument: Computed output size would be negative: -34 [input_size: 37, effective_filter_size: 72, stride: 1]
         [[node import/FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/AvgPool (defined at D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\modelzoo\vision_base.py:200) ]]
         [[Mean/_29]]
0 successful operations.
0 derived errors ignored. 

[colah]

Hello! It sounds like your parameterization may be too small for the neuron you are trying to visualize.

Try making your parametrization larger (eg. param_f = lambda: param.image(larger_numer)).

[tul-urte]

I had 128, and tried up to 2048, with the declared error varying:

  (0) Invalid argument: Incompatible shapes: [1,16,490,490] vs. [1,16,419,363]
         [[node import/FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/mul (defined at D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\modelzoo\vision_base.py:200) ]]
         [[Mean/_27]]
  (1) Invalid argument: Incompatible shapes: [1,16,490,490] vs. [1,16,419,363]
         [[node import/FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/mul (defined at D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\modelzoo\vision_base.py:200) ]]
0 successful operations.
0 derived errors ignored.

There's a bit more stacktrace which indicates the problem occurs while loading the frozen graph (i.e. lucid\modelzoo\vision_base.py: line 200, in import_graph, below):

Original stack trace for 'import/FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/mul':
  File "build_sprites.py", line 35, in <module>
    model_loader=lambda steps: graph_models.model_for_version(
  File "../lib\vis.py", line 898, in build_sprites
    loss_logger=lambda l,t,s: loss_logger(
  File "../lib\vis.py", line 209, in get_visualizations_and_losses
    T = render.make_vis_T( model, objective_f, param_f, optimizer, transforms )
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\optvis\render.py", line 179, in make_vis_T
    T = import_model(model, transform_f(t_image), t_image)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\optvis\render.py", line 257, in import_model
    model.import_graph(t_image, scope=scope, forget_xy_shape=True, input_map=input_map)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\lucid-0.3.9-py3.7.egg\lucid\modelzoo\vision_base.py", line 200, in import_graph
    self.graph_def, {self.input_name: t_prep_input}, name=scope)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\importer.py", line 443, in import_graph_def
    _ProcessNewOps(graph)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\importer.py", line 236, in _ProcessNewOps
    for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\ops.py", line 3751, in _add_new_tf_operations
    for c_op in c_api_util.new_tf_operations(self)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\ops.py", line 3751, in <listcomp>
    for c_op in c_api_util.new_tf_operations(self)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\ops.py", line 3641, in _create_op_from_tf_operation
    ret = Operation(c_op, self)
  File "D:\Miniconda3\envs\tf-gpu2\lib\site-packages\tensorflow\python\framework\ops.py", line 2005, in __init__
    self._traceback = tf_stack.extract_stack()

But then I don't understand why varying the scale (as you suggested) would change the error details.

[tul-urte]

I downloaded the trained MobileNet V3 graph file "Small dm=0.75 (float)" from here (https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet), extracted the files into my "D:/temp", and ran the following code:

import tensorflow as tf
import lucid.optvis.render as render
from lucid.modelzoo.vision_base import Model

class SSDMobilenetV3( Model ):
    def __init__( self, graph_path  ):        
        self.model_path = graph_path
        self.input_name = "input"
        self.image_shape = [ 224, 224, 3 ]
        self.image_value_range = ( -1, 1 )
        super().__init__()

model = SSDMobilenetV3( "D:/temp/v3-small_224_0.75_float/v3-small_224_0.75_float.pb" )
model.load_graphdef()

#model.show_graph()

_ = render.render_vis( model, "MobilenetV3/expanded_conv_6/output:0" )

There's a fair bit of stacktrace, but the key errors are:

LookupError: gradient registry has no entry for: AddV2

and

LookupError: No gradient defined for operation 'import/MobilenetV3/expanded_conv_6/squeeze_excite/Conv_1/add' (op type: AddV2)

I also tried using the V3_SMALL_MINIMALISTIC definition in "mobilenet_v3.py" (registering a new feature extractor) to train a test model. This is essentially the same model but without the "squeeze_excite" insertions.

The above code ran fine on the new model, rendering an image.

This leads me to believe that the probem is in the "squeeze_excite" implementation (in mobilenet/conv_blocks.py).