LIFRate outputs 0 after a single epoch

[arvoelke]

import numpy as np
import matplotlib.pyplot as plt

import nengo
import nengo_dl
import tensorflow as tf

u = np.random.randn(100, 1, 1)
dt = 0.001

with nengo.Network() as model:
    stim = nengo.Node(output=nengo.processes.PresentInput(
        u, presentation_time=dt))
    x = nengo.Ensemble(100, 1,
        neuron_type=nengo.LIFRate())  # <-- HERE
    nengo.Connection(stim, x, synapse=None)
    p = nengo.Probe(x, synapse=None)

inputs = {stim: u}
targets = {p: u}
opt = tf.train.MomentumOptimizer(
    learning_rate=1e-13, momentum=0.1, use_nesterov=True)

with nengo_dl.Simulator(model, minibatch_size=1, dt=dt) as sim:
    sim.train(inputs, targets, opt, n_epochs=1)  # <-- HERE
    sim.run(len(u)*dt)

plt.figure()
plt.plot(sim.trange(), sim.data[p].squeeze())
plt.plot(sim.trange(), u.squeeze(), linestyle='--')
plt.show()

Initially this is a communication channel. After a single epoch (with essentially 0 learning rate) the ensemble outputs a flat 0. Commenting out either of the lines labeled # <-- HERE makes the problem go away. Changing the neuron type to Sigmoid or LIF also makes the problem go away.

[drasmuss]

Definitely something weird going on here, I'll dig into it.

[drasmuss]

Can you try out #27 and see if it works for you?

The problem was that you were getting nans when computing the gradient of LIFRate, which TensorFlow was then silently converting into zero output for some reason. Although LIFRate isn't technically differentiable, we should still be able to compute the gradient without getting nans, which is what #27 does. But just for your own future work, I'd recommend using nengo_dl.SoftLIFRate instead, it'll work better during training.

[arvoelke]

Thanks! This makes sense and works for me. For some reason I had it in the back of my head that nengo_dl would automatically switch the LIFRate for SoftLIFRate to do the optimization and then switch it back. Maybe a way to do this would be useful. If not, then a warning and/or some clip at a maximum value would help this from recurring.

Relatedly... I think I'm misunderstanding how the trainable config works. I was using this to disable the LIFRate populations, but that doesn't seem to prevent them from creating problems.

import numpy as np
import matplotlib.pyplot as plt

import nengo
import nengo_dl
import tensorflow as tf

u = np.random.randn(100, 1, 1)
dt = 0.001

with nengo.Network() as model:
    stim = nengo.Node(output=nengo.processes.PresentInput(
        u, presentation_time=dt))
    x = nengo.Ensemble(100, 1, neuron_type=nengo.LIFRate())
    y = nengo.Ensemble(100, 1, neuron_type=nengo_dl.SoftLIFRate())

    nengo_dl.configure_settings(trainable=True)
    model.config[x].trainable = False

    #nengo_dl.configure_settings(trainable=False)
    #model.config[y].trainable = True

    nengo.Connection(stim, x, synapse=None)
    nengo.Connection(x, y, synapse=None)
    p = nengo.Probe(y, synapse=None)

inputs = {stim: u}
targets = {p: u}
opt = tf.train.MomentumOptimizer(
    learning_rate=1e-13, momentum=0.1, use_nesterov=True)

with nengo_dl.Simulator(model, minibatch_size=1, dt=dt) as sim:
    sim.train(inputs, targets, opt, n_epochs=1)
    sim.run(len(u)*dt)

plt.figure()
plt.plot(sim.trange(), sim.data[p].squeeze())
plt.plot(sim.trange(), u.squeeze(), linestyle='--')
plt.show()

If you switch the two trainable lines that are commented, it goes from being broken (on master, without #27) to working. But I thought that neither approach would need to differentiate the tuning curves for the x population.

[drasmuss]

The trainable flag sets the parameters associated with that object to be (non)trainable, but you still need to backpropagate the gradient through those objects (to get the gradient for any preceding objects, which may still be trainable). So in your example, even if you set x to be not trainable, you still get nans when you propagate the gradient through x, which results in nans on the connection weights from stim -> x. In the second case (configure_settings(trainable=False)) you're also setting that stim->x connection to be non-trainable, so you don't end up with nan weights.

[drasmuss]

For some reason I had it in the back of my head that nengo_dl would automatically switch the LIFRate for SoftLIFRate to do the optimization and then switch it back. Maybe a way to do this would be useful.

Yeah that should be possible. I probably wouldn't make it automatic by default, but could probably work up a flag or helper function that'd make that happen.

[arvoelke]

Thanks for the explanation. To check my understanding, backprop is modifying the scalar transform from stim -> x to 0? Or could it even be modifying some other weights (e.g., gains, biases, encoders of x)?

[drasmuss]

It's actually setting the transform to nan in this case (so the output of the network is nan, but I guess TensorFlow has some logic to convert that to zero).

The gains/biases/encoders are all associated with the trainability of x, so if you set x to be non-trainable then they will be fixed. You can use x.neurons to separately control the trainability of the biases, but you can't separate gains and encoders since those are actually just combined into one parameter in Nengo, the scaled_encoders. More details in the documentation (in case anyone comes across this discussion in the future).

[arvoelke]

Cool, that clears it all up. Thanks for the quick replies. Feel free to close whenever.