search

UKPLab / sentence-transformers

Multilingual Sentence & Image Embeddings with BERT
https://www.SBERT.net
Apache License 2.0
14.38k stars 2.4k forks source link

How to add noise in embedding layer？ #563

Open yysirs opened 3 years ago

yysirs commented 3 years ago

I want to improve the generalization of the model

nreimers commented 3 years ago

Dropout is used in BERT etc.

What you could do is to drop out whole word or word-pieces from the input. But this would require that you modify your training data so that the sentences are incomplete (some random words are deleted)

Sauce16 commented 4 months ago

hi @yysirs could you share me the code? coz I am new and I am not sure how to add noise in the embedding layer