Cannot save/load models with custom losses

[cshimmin]

I am trying to save models which have custom loss functions that are added to the model using Model.add_loss(). This is NOT the same issue which has already been seen several times, where you have to pass custom_objects=... to load_model(); in fact, when using add_loss, I do not include any loss function when calling Model.compile().

Here is a brief script that can reproduce the issue:

import keras
import keras.backend as K
import numpy as np

# make some trivial model with a single dense layer
model_input = keras.layers.Input((1,))
model_output = keras.layers.Dense(100)(model_input)
model = keras.models.Model(model_input, model_output)

# define the loss function directly to be the mean square activation of the model output
model.add_loss(K.mean(K.square(model_output)))

model.compile(optimizer='adam')

# fit on random inputs. Note that no target y-values are specified, since the
# loss of the model is completely defined by the output activation
model.fit(np.random.normal(size=100), None)

model.save('test.h5')

keras.models.load_model('test.h5')

Everything in this script works as expected, except for the last line. Instead of loading the model, I get an error:

ValueError: When passing a list as loss, it should have one entry per model outputs. The model has 1 outputs, but you passed loss=[]

Of course, it should be noted that this model is a toy example and is not doing something super interesting (it will just learn to output zeros). One actual application where I regularly have this problem is with variational autoencoders, where the reconstruction and KL losses are added in this manner, and there is no explicit comparison of y_pred and y_test, since the loss in such models is also completely defined by the input, as well as depending on other things such as internal layer outputs for the encoder means and variances.

Is there some way with Keras to save/load these kinds of models, where the loss is defined explicitly as some kind of tensor expression?

---

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.

Thank you!

• [x] Check that you are up-to-date with the master branch of Keras. You can update with: pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• [x] Check that your version of TensorFlow is up-to-date. The installation instructions can be found here.

• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[hyeokhyen]

so this question is left never answered ...?

[xing-w]

Use save_weights to save the model. When loading, build&compile a model first, then use load_weights. Maybe you could compile the model after load weights. I didn't try.

[cshimmin]

@xing-w, the issue I describe is not that save_weights does not work as expected. The issue is specifically regarding saving the model. Of course I can just re-instantiate the exact same keras model and load in the weights. But for this to be practically useful one would need to also export a bunch of metadata about how to re-build the model to a separate file. Not to mention that saving/loading weights does not solve the issue of saving the optimizer state. All of this is exactly what Keras' Model.save() is supposed to do, except that it does not work as expected, as reported here.

[ttvand]

I ended up writing a custom Callback in a VAE setup to save the encoder and decoder separately. This may be relevant to your setup too.

[QCaudron]

Is this a solution ? https://stackoverflow.com/questions/48373845/loading-model-with-custom-loss-keras

[ljsun]

so this question no answer? is bug?

[QCaudron]

The custom_objects kwarg to model saving and loading worked great for me.

[rocassius]

Would you expand on how you used the custom_objects kwarg please, QCaudron?

[QCaudron]

See docs here ( under "Handling custom layers (or other custom objects) in saved models" ) : https://keras.io/getting-started/faq/

Here's some of my code that works well.

# Loss function : the Jaccard index
def jaccard_coef(y_true, y_pred):
    """
    The Jaccard index, where 1 is the perfect overlap.
    """
    y_true_flat = K.flatten(y_true)
    y_pred_flat = K.flatten(y_pred)
    intersection = K.sum(y_true_flat * y_pred_flat)
    union = K.sum(y_true_flat) + K.sum(y_pred_flat) - intersection
    return -intersection / (union + 1e-6)

# Loading the model
model = load_model(
    "unet_profond.h5", 
    custom_objects={"jaccard_coef": jaccard_coef}
)

Your function must be defined in scope at load time, so you can refer to it. Saving is the same.

[ihalage]

I hope people encounter this issue with VAEs (specifically when loading the vae model in the keras example vae code). Please see my answer here for a possible solution and example implementation.

[GPla]

@ihalage This is no fix for the issue. @cshimmin already mentioned this approach and it is not practical.

[ihalage]

@GPla First, this is not a fix for this particular issue, it is a workaround to save and load a model, specifically in VAEs (which is mentioned in my comment). Second, I have practically implemented and tested it.