mean is not called on categorical_crossentropy

[xuzhuoran0106]

I notice that mean is not called on categorical_crossentropy

def categorical_crossentropy(y_true, y_pred): '''Expects a binary class matrix instead of a vector of scalar classes. ''' return K.categorical_crossentropy(y_pred, y_true)

but on binary_crossentropy, mean is called.

def binary_crossentropy(y_true, y_pred): return K.mean(K.binary_crossentropy(y_pred, y_true), axis=-1)

Could someone explain why to me?

Thanks

[jaikumarm]

@xuzhuoran0106 where are you seeing this code. they both should be mean.

See this file: https://github.com/fchollet/keras/blob/master/keras/metrics.py

def categorical_crossentropy(y_true, y_pred): '''Expects a binary class matrix instead of a vector of scalar classes. ''' return K.mean(K.categorical_crossentropy(y_pred, y_true))

def binary_crossentropy(y_true, y_pred): return K.mean(K.binary_crossentropy(y_pred, y_true))

[xuzhuoran0106]

@jaikumarm Thanks for reply.

I see the code form objectives.py https://github.com/fchollet/keras/blob/master/keras/objectives.py#L36

Also, I have seen the mean version in https://github.com/fchollet/keras/blob/master/keras/metrics.py.

so, I think the code in objectives.py in used to compute gradient, and what in metrics.py is used to calculate log information.

please see code in _make_train_function, where self.total_loss is argument of optimizer.get_updates, but self.metrics_tensors not.

https://github.com/fchollet/keras/blob/ea561ba6d87879b9a67a7454e1d29777fb59132d/keras/engine/training.py#L686

def _make_train_function(self):

    if not hasattr(self, 'train_function'):
        raise Exception('You must compile your model before using it.')
    if self.train_function is None:
        if self.uses_learning_phase and type(K.learning_phase()) is not int:
            inputs = self.inputs + self.targets + self.sample_weights + [K.learning_phase()]
        else:
            inputs = self.inputs + self.targets + self.sample_weights

        # get trainable weights
        trainable_weights = collect_trainable_weights(self)
        training_updates = self.optimizer.get_updates(trainable_weights, self.constraints, self.total_loss)
        updates = self.updates + training_updates

        # returns loss and metrics. Updates weights at each call.
        self.train_function = K.function(inputs,
                                         [self.total_loss] + self.metrics_tensors,
                                         updates=updates,
                                         **self._function_kwargs)

[George-Zhu]

And the contrastive loss in keras/examples/mnist_siamese_graph.py calls K.mean() within one batch, which is inconsistent with the categorical_crossentropy() loss.

[jasonwbw]

@xuzhuoran0106 A mean function will be called inside the weighted function of training.py.