Probing spiking activity from a particular output in the converter

[arvoelke]

(Feature Request)

import nengo
import nengo_dl

import tensorflow as tf
from tensorflow.keras.layers import Input, Dense

inp = Input(4)
dense = Dense(8, activation="relu", name="dense")(inp)
out = Dense(1, name="out")(dense)

model = tf.keras.Model(inputs=inp, outputs=[out, dense])

converter = nengo_dl.Converter(
    model,
    swap_activations={
        tf.keras.activations.relu: nengo.SpikingRectifiedLinear(),
    },
    synapse=0.005,
)

for conn in converter.net.all_connections:
    if conn.synapse is not None:
        print(conn, conn.synapse)

for probe in converter.net.all_probes:
    if probe.synapse is not None:
        print(probe, probe.synapse)

=>

<Connection from <Neurons of <Ensemble "dense.0">> to <Node "out.0">> Lowpass(tau=0.005)
<Probe of 'output' of <Neurons of <Ensemble "dense.0">>> Lowpass(tau=0.005)

Here we get the same synapse (Lowpass(0.005)) on both the connection from "dense" and the probe from "dense". But I would like to probe the unfiltered spiking activity of dense (hence why I am including it as an additional output in tf.keras.Model.

My current work-around is to add converter.outputs[dense].synapse = None after nengo_dl.Converter is invoked. It would be nice if there was a way to do this within the call to the converter and/or some documentation about where synapses are placed and how to control them more carefully.

[drasmuss]

I'd recommend just using the Nengo API for this (like nengo.Probe(converter.layers[dense])), rather than trying to do things through the converter API. Note that you don't need dense to be an output of the Keras model, you can probe any arbitrary layer within the model.

[arvoelke]

I still need the other Keras model output for other reasons (outside of nengo_dl). So would your solution be as efficient as mine or would it effectively double the amount of output data being generated during inference?

[drasmuss]

Yeah you're creating a new Probe, so it would have its own output data. If you're going to be generating the dense probe automatically anyway, then yeah I'd just go with the converter.outputs[dense].synapse = None solution. That seems relatively clean to me, I think that the benefits of trying to add a second way to configure synapses through the Converter call would provide relatively minor benefits for the added code complexity. In general I think that we want the Converter to be relatively lightweight, and then be directing users to the Nengo API if they want more control (rather than trying to create a large conceptual/API layer in between Keras and Nengo).

[arvoelke]

Makes sense. I'll mark this closed, as it's resolved from my perspective then. Thanks!