laplacian train

[ttll0363]

Hi, thanks for your clear code, Can the Laplacian train code be released? I think there is some difference in the training process for Laplacian, right?

[FMILab]

The only difference is in the loss function. The negative log likelihood of the Laplace distribution is used to construct the loss function:

def laplacian_loss(y_true, y_pred): mask_true = y_true[:,:,:,0] mask_pred = y_pred[:,:,:,0] mask_scale_pred = y_pred[:,:,:,1] loss_4 = (tf.divide(K.abs(mask_true - mask_pred), mask_scale_pred + 1e-9) + K.log(mask_scale_pred + 1e-9)) return loss_4

Note that the 1e-9 term is not in the log likelihood term. This is only included to prevent dividing by zero. If you use the Laplacian distribution, make sure to also update where you select the loss function for the model:

BCNN_train.compile(optimizer=adm, loss=laplacian_loss)

Similarly, for the Gaussian distribution, the loss function becomes:

def gauss_loss(y_true, y_pred): mask_true = y_true[:,:,:,0] mask_pred = y_pred[:,:,:,0] mask_scale_pred = y_pred[:,:,:,1] loss_5 = K.mean(tf.divide(tf.square(mask_true - mask_pred), tf.square(mask_scale_pred)*2 + 1e-9) + K.log(mask_scale_pred + 1e-9)) return loss_5

[ttll0363]

Thanks for your clarification! Also, I know that the output of a Laplace or Gaussian distribution should be two-channel. One question I have is, can I still use sigmoid for the activation function in the last layer?

[FMILab]

We have found the best results if we use an elu activation for the "mean" channel and a sigmoid for the "standard deviation" channel. It is important to make sure the standard deviation is nonnegative, making sigmoid a good choice. I would anticipate that ReLu would give similar performance for the "mean" channel. Sigmoid could also work for the "mean" channel, but we have seen that it leads to less consistent training.

[ttll0363]

thanks for your reply! In addition to the loss function, this means that if I want to use Gaussian instead of Bernoulli, I need to change the output format in the model, right?