Questions on semantic similarity

[Tauvic]

Im working on a dutch bible project and therefore interested in semantic similarity. Are there any plans to make a sentence similarity model. The only models I found that support semantic similarity in dutch are multi lingual models.

• sentence-transformers/xlm-r-100langs-bert-base-nli-stsb-mean-tokens
• sentence-transformers/distilbert-multilingual-nli-stsb-quora-ranking

My plan for now is:

• Find some model that supports Dutch
• Train it on sentence similarity (how, where to get a decent dataset)
• There are some parallel bible translations that can be used as a start but there are no similarity scores
• Evaluate the results

Im also looking for datasets to train a model on that. The Bertje model also does not have a model trained on sentence similarity Any suggestions that can help me?

[twinters]

Hi Tauvic,

We have not trained a sentence similarity model ourselves. However, existing implementations for finetuning such a sentence similarity using RoBERTa should also be easily transferable to RobBERT by just loading in our model instead, and providing your own Dutch dataset. You plan sounds like the right plan to achieve this with RobBERT if you somehow find similarity scores for them. You could potentially already get started by (automatically) translating existing sentence similarity corpora if none exist for Dutch.

Hope this helps, and good luck on your project!

[Tauvic]

Thanks for the response. I will look into it. But for now i have switched to another project on driving safety: https://github.com/Tauvic/DriverAwareness