DeepLift slow-down... - Githubissues

Thanks so much for the great package. It helps to have a single interface for so many explanation techniques. I'm trying to use the DeepLift analyzer, and finding that it slows down for each query done against an analyzer. That is, if I train a Keras model on the MNIST data, and then build an analyzer as follows:

model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
model_name = 'delme_tmp_model.h5'
if SPEED_UP_BY_REFRESHING_MODEL_EACH_TIME:
    model.save(model_name)

analyzer = innvestigate.create_analyzer('deep_lift.wrapper', model_wo_softmax,
                                        **{'reference_inputs': 128, 'nonlinear_mode': 'reveal_cancel'})
analyzer.fit(train_x, batch_size=256, verbose=1)

... and then if I go through the test data and analyze them, the DeepLift analyzer gets slower on each new image (22s, 30s, 38s, 46s, 69s, 91s for the 1st, 2nd, 3rd, ... images):

for i, img in enumerate(test_x):
    add_dim_img = np.array(img)
    add_dim_img = add_dim_img[np.newaxis, :, :, :]
    # Speed-up work-around from below goes here
    analysis = analyzer.analyze(add_dim_img)
    current = time.time()
    print('Analyzed One Image: %lf' % (current - last))
    last = current
    sys.stdout.flush()

A colleague of mine found the following work-around that makes the each run a bit slower than the first run without the work-around, but there doesn't appear to be any slow-down otherwise:

del model
if model_wo_softmax is not None:
    del model_wo_softmax
keras.backend.clear_session()
tf.compat.v1.reset_default_graph()
model = keras.models.load_model(model_name) # I've saved the built model out to a file with model.save(model_name)
model_wo_softmax = iutils.keras.graph.model_wo_softmax(model)
analyzer = innvestigate.create_analyzer('deep_lift.wrapper', model_wo_softmax,
                                        **{'reference_inputs': 128, 'nonlinear_mode': 'reveal_cancel'})
analyzer.fit(train_x, batch_size=256, verbose=1)

I don't know if this can help you understand what might be causing the slow-down, but figured I'd report it!

albermax / innvestigate

DeepLift slow-down... #194