keras layer

[shainaraza]

Can I use keras layer with bert embeddings from sentence-transformers?

[
    merged = Dot(name = 'dot_product', normalize = True, axes = 1)([h_embeddings, s_embeddings])

    # Reshape to be a single number (shape will be (None, 1))
    merged = Reshape(target_shape = [1])(merged)

    # If classifcation, add extra layer and loss function is binary cross entropy
    if classification:
        merged = Dense(1, activation = 'sigmoid')(merged)
        model = Model(inputs = [headline, snippet], outputs = merged)
        model.compile(optimizer = 'Adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

    # Otherwise loss function is mean squared error
    else:
        model = Model(inputs = ([headline, snippet]), outputs = merged)
        model.compile(optimizer = 'Adam', loss = 'mse')

    return model](url)

[nreimers]

Hi @shainaraza This library works with pytorch, keras is sadly based on tensorflow.

You could create the embeddings and use them as inputs for keras.

Or you use the description from huggingface and convert the fine-tuned BERT to tensorflow and then integrate it into keras.

Best Nils Reimers

[shainaraza]

Thanks a lot for prompt reply and indeed useful