lstm error: KeyError: 'sentence_lengths'

[MingchangLi]

hi @nreimers , it's a nice repo. When I read your code in training_stsbenchmark_bilstm.py. I want to test the performance of bert + bilstm, but maybe there is a bug in lstm. I have read all issues about lstm in this repo, it looks like some of us met some error. Could you tell me how to fix? Although I have known that bert + bilstm cannot outperform than bert + pool, but I just want to prove the result by experiment. I will be grateful for your kindly help.

Here is my code below:

word_embedding_model = models.Transformer('bert-base-chinese', max_seq_length=256) lstm = models.LSTM(word_embedding_dimension = word_embedding_model.get_word_embedding_dimension(), hidden_dim = 384, num_layers = 1, dropout = 0.5, bidirectional = True) pooling_model = models.Pooling(lstm.get_word_embedding_dimension()) model = SentenceTransformer(modules=[word_embedding_model, lstm, pooling_model]) sentences = ['This framework generates embeddings for each input sentence','Sentences are passed as a list of string.', 'The quick brown fox jumps over the lazy dog.'] sentence_embeddings = model.encode(sentences) print(sentence_embeddings)<\code>

And here is the error:

[XueQZ]

Same question as #834 . You need to create a new layer class that extracts the sentence lengths from the BERT tokenizer / the BERT attention mask.