Keras: cannot identify custom loss function name/type

[aldahdooh]

I am building a model with two outputs with custom loss function. Then I am trying to apply an attack on the model using KerasClassifier. but I have the following error (I guess that Kera wrapper coudn't recognize the the loss type/name).

To Reproduce

1. run the code

   
   from __future__ import print_function

import keras import numpy as np import pickle import json import os import yaml from keras import backend as K from keras import optimizers from keras import regularizers from keras.datasets import cifar10 from keras.engine.topology import Layer from keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Concatenate from keras.layers import Dense, Dropout, Activation, Flatten, Input from keras.layers import Lambda from keras.models import Model from keras.models import Sequential from keras.preprocessing.image import ImageDataGenerator from art.classifiers import KerasClassifier from art.attacks import FastGradientMethod import matplotlib.pyplot as plt

function

def normalize(X_train, X_test): X_train = X_train / 255.0 X_test = X_test / 255.0 return X_train, X_test

function

def load_cifar10_data(): num_classes = 10 (x_train, y_train), (x_test, y_test_label) = cifar10.load_data() x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train, x_test = normalize(x_train, x_test)

y_train = keras.utils.to_categorical(y_train, num_classes + 1)
y_test = keras.utils.to_categorical(y_test_label, num_classes + 1)
return (x_train, y_train), (x_test, y_test)

function

def build_cifar10_model(input_shape): weight_decay = 0.0005 input = Input(shape=input_shape) num_classes = 10

task0 = Conv2D(64, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(input)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(64, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)
task0 = MaxPooling2D(pool_size=(2, 2))(task0)

task0 = Conv2D(128, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(128, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)
task0 = MaxPooling2D(pool_size=(2, 2))(task0)

task0 = Conv2D(256, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(256, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(256, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)
task0 = MaxPooling2D(pool_size=(2, 2))(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)
task0 = MaxPooling2D(pool_size=(2, 2))(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task0 = Conv2D(512, (3, 3), padding='same', kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)
task0 = MaxPooling2D(pool_size=(2, 2))(task0)

task0 = Flatten()(task0)
task0 = Dense(512, kernel_regularizer=regularizers.l2(weight_decay))(task0)
task0 = Activation('relu')(task0)
task0 = BatchNormalization()(task0)

task1 = Dense(num_classes, activation='softmax')(task0)

task2 = Dense(512, kernel_regularizer=regularizers.l2(weight_decay))(task0)
task2 = Activation('relu')(task2)
task2 = BatchNormalization()(task2)
task2 = Dense(1, activation='sigmoid')(task2)

output1 = Concatenate(axis=1, name="output1")([task1, task2])
output2 = Dense(num_classes, activation='softmax', name="output2")(task0)

model = Model(inputs=input, outputs=[output1, output2])
return model

function

def loss_fun(y_true, y_pred): num_classes = 10 loss = K.categorical_crossentropy( K.repeat_elements(y_pred[:, -1:], num_classes, axis=1) * y_true[:, :-1], y_pred[:, :-1]) return loss

function

def my_generator(func, x_train, y_train, batch_size): while True: res = func(x_train, y_train, batch_size).next() yield [res[0], [res[1], res[1][:, :-1]]]

variables

(x_train, y_train), (x_test, y_test) = load_cifar10_data()

variables for model

num_classes = 10 weight_decay = 0.0005 basic_dropout_rate=0.3 mc_dropout_rate = 0 #K.variable(value=0) input_shape = x_train.shape[1:] filename = 'test_selective.h5'

cifar_model = build_cifar10_model(input_shape)

variables for training

alpha = 0.5 batch_size = 128 maxepoches = 5 learning_rate = 0.1 lr_decay = 1e-6 lr_drop = 25 def lr_scheduler(epoch): return learning_rate * (0.5 ** (epoch // lr_drop)) reduce_lr = keras.callbacks.LearningRateScheduler(lr_scheduler)

data augmentation

datagen = ImageDataGenerator( featurewise_center=False,
samplewise_center=False,
featurewise_std_normalization=False, samplewise_std_normalization=False, zca_whitening=False,
rotation_range=15, width_shift_range=0.1,
height_shift_range=0.1, horizontal_flip=True,
vertical_flip=False) datagen.fit(x_train)

sgd = optimizers.SGD(lr=learning_rate, decay=lr_decay, momentum=0.9, nesterov=True) cifar_model.compile(loss={'output1': loss_fun, 'output2': 'categorical_crossentropy'}, loss_weights=[alpha, 1 - alpha], optimizer=sgd, metrics=['accuracy'])

historytemp = cifar_model.fit_generator(my_generator(datagen.flow, x_train, y_train, batch_size=batch_size),

steps_per_epoch=x_train.shape[0] // batch_size,

epochs=maxepoches, callbacks=[reduce_lr],

validation_data=(x_test, [y_test, y_test[:, :-1]]))

loss_all, loss_out1, loss_out2, out1_acc, out2_acc = cifar_model.evaluate(x_test, [y_test, y_test[:,:-1]])

cifar_model.saveweights("checkpoints/{}".format('wts' + filename))

cifar_model.save("checkpoints/{}".format(filename))

model = build_cifar10_model(input_shape) model.loadweights("checkpoints/{}".format('wts' + filename)) model.compile(loss={'output1': loss_fun, 'output2': 'categorical_crossentropy'}, loss_weights=[alpha, 1 - alpha], optimizer=sgd, metrics=['accuracy']) model.summary() loss_all, loss_out1, loss_out2, out1_acc, out2_acc = model.evaluate(x_test, [y_test, y_test[:,:-1]])

classifier = KerasClassifier(model=model, clip_values=(0, 1)) attack_fgsm = FastGradientMethod(classifier=classifier, eps=0.3) x_test_adv = attack_fgsm.generate(x_test)

loss_adv_all, loss_adv_out1, loss_adv_out2, out1_adv_acc, out2_adv_acc = model.evaluate(x_test_adv, [y_test, y_test[:,:-1]]) perturbation = np.mean(np.abs((x_test_adv - x_test))) print('Accuracy on adversarial test data: {:4.2f}%'.format(out1_adv_acc * 100)) print('Average perturbation: {:4.2f}'.format(perturbation))

plt.figure() plt.imshow(x_test_adv[0,:,:,0]) plt.show()

print('Done....')

2. See error
`Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/aaldahdo/.vscode/extensions/ms-python.python-2020.5.80290/pythonFiles/lib/python/debugpy/no_wheels/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/aaldahdo/.vscode/extensions/ms-python.python-2020.5.80290/pythonFiles/lib/python/debugpy/no_wheels/debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/aaldahdo/.vscode/extensions/ms-python.python-2020.5.80290/pythonFiles/lib/python/debugpy/no_wheels/debugpy/../debugpy/server/cli.py", line 267, in run_file
    runpy.run_path(options.target, run_name=compat.force_str("__main__"))
  File "/usr/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/aaldahdo/adv_dnn/multi_output.py", line 196, in <module>
    classifier = KerasClassifier(model=model, clip_values=(0, 1))
  File "/home/aaldahdo/adv_dnn/venv/lib/python3.8/site-packages/art/classifiers/keras.py", line 101, in __init__
    self._initialize_params(model, use_logits, input_layer, output_layer)
  File "/home/aaldahdo/adv_dnn/venv/lib/python3.8/site-packages/art/classifiers/keras.py", line 184, in _initialize_params
    loss_function = getattr(k, self._model.loss.__name__)
AttributeError: 'dict' object has no attribute '__name__'`

**Expected behavior**
I expected to have the model wrapped.

**System information (please complete the following information):**
 - Ubuntu 18
 - Python version 3.8
 - ART version  1.2
 - Keras version 2.3.1

[beat-buesser]

Hi @ahmedomer6 Thank you very much for using ART and raising this question! The current version of KerasClassifier does not support custom loss functions, only part of the loss functions defined by Keras. The other classifiers in ART like TensorFlowV2Classifier are more flexible in terms of loss functions.

[aldahdooh]

Hi @ahmedomer6 Thank you very much for using ART and raising this question! The current version of KerasClassifier does not support custom loss functions, only part of the loss functions defined by Keras. The other classifiers in ART like TensorFlowV2Classifier are more flexible in terms of loss functions.

Many thanks for your reply @beat-buesser . I am not using TensorFlowV2Classifier since when I am saving a callable model and then load it again I couldn't generate attacks with generate function. It says not callable. Hence, I moved to Keras.

[beat-buesser]

@ahmedomer6 Did you try making your model callable? ART's example/get_started_tensorflow_v2.py shows how to use a callable model.

[aldahdooh]

@beat-buesser Yes, I did the same. it works fine. But if the model in the example saved and loaded again, you cannot generate attacks.

[beat-buesser]

How did you save the model and why can it not run a new attack?

[aldahdooh]

@ I used the following steps 1-create KerasModel() 2-train it 3-save weights 4-create new KerasModel() 5-load weights

[beat-buesser]

Thank you. What is the error message that you receive after running an attack on the new model after step 5?

[aldahdooh]

I tracked the error below. It comes from loss object since i defined my own loss function

The error art/classifiers/tensorflow.py", line 947, in loss_gradient loss = self._loss_object(y, predictions) TypeError: 'list' object is not callable

[beat-buesser]

Would it be possible for you to post a code example with a small model form the ART examples that reproduces this error? I would be interested to find out why this is happening.