kyle-dorman / bayesian-neural-network-blogpost

Building a Bayesian deep learning classifier
https://medium.com/towards-data-science/building-a-bayesian-deep-learning-classifier-ece1845bc09
481 stars 105 forks source link

Unknown entry in loss dictionary: "logits_variance". #8

Open leedrake5 opened 4 years ago

leedrake5 commented 4 years ago

model.compile(optimizer=optimizers.Adam(lr=1e-3, decay=0.001), loss={'logits_variance': bayesian_categorical_crossentropy(100, 10), 'softmax_output': 'categorical_crossentropy'}, metrics={'softmax_output': metrics.categorical_accuracy}, loss_weights={'logits_variance': .2, 'softmax_output': 1.})

returns the "logits_variance" issue in the title, or the following:

TypeError: Expected float32, got <tile.Value ZerosLike FLOAT32(<tile.Value Ceiling UINT64()>, -7)> of type 'Value' instead. ...after many file path dumps in the error.

I've tried using the code here, but I keep running into this. Scanning the code, I'm unclear of where this is defined. Any advice would be helpful. I've tried defining my own model, I've also tried generating a model as follows:

model=create_bayesian_model('resnet50', (400, 400, 3), 2)

leedrake5 commented 4 years ago

Been working on this for a couple days, I think I've resolved part of the error by defining terms. But I keep running into this once an epoch opens for a bnn:

ValueError: Error when checking input: expected input_8 to have 2 dimensions, but got array with shape (32, 400, 400, 3)

I think the problem is in the header loading from resnet50 in the original data used. Which is likely due to an argument or default changing in tensorflow/keras versions since this project was last updated 2 years ago. Will report here if I find a solution.

Update - this is the case. Just using the resnet50() with input dimensions as the only argument throws this warning:

UserWarning: The output shape of ResNet50(include_top=False) has been changed since Keras 2.2.0. warnings.warn('The output shape of ResNet50(include_top=False)

The code is as follows:

def resnet50(input_shape):
    input_tensor = Input(shape=input_shape)
    base_model = ResNet50(include_top=False, weights='imagenet', input_tensor=input_tensor)
    # freeze encoder layers to prevent over fitting
    for layer in base_model.layers:
        layer.trainable = False
    output_tensor = Flatten()(base_model.output)
    return Model(inputs=input_tensor, outputs=output_tensor)

So include_top is what needs to be updated.