nengo / nengo-dl

Deep learning integration for Nengo
https://www.nengo.ai/nengo-dl
Other
88 stars 22 forks source link

Extra loss functions in loss list are ignored #123

Closed arvoelke closed 4 years ago

arvoelke commented 4 years ago
import nengo
import nengo_dl
import numpy as np
import tensorflow as tf

ignored_loss = lambda y_true, y_pred: 0 / 0  # raise ZeroDivisionError

with nengo.Network() as net:
    p = nengo.Probe(nengo.Node(1))

with nengo_dl.Simulator(net) as sim:
    sim.compile(loss=[nengo_dl.losses.Regularize(), ignored_loss])
    sim.fit(n_steps=1, y=np.zeros((1, 1, 1)), epochs=1)

Expected behaviour: this should raise an error or warning saying that extra elements in the loss list are not being used. We can see that ignored_loss is being ignored since no ZeroDivisionError is raised; if we make it the first element in the loss list then we do get the zero division error.

Context: I was trying to do this in the hope that it might somehow weight together the two loss functions (normally in Keras one can easily add extra loss functions like regularization to other parts of the network). For reference, to do that the correct way, see the pattern in this unit test: https://github.com/nengo/nengo-dl/blob/7363bc342bd9b980e04f9ed827f25ad8620d0dcd/nengo_dl/tests/test_objectives.py#L38-L76

drasmuss commented 4 years ago

A warning was added in https://github.com/nengo/nengo-dl/pull/139