search

UKPLab / sentence-transformers

Multilingual Sentence & Image Embeddings with BERT
https://www.SBERT.net
Apache License 2.0
14.38k stars 2.39k forks source link

reproduce Augmented SBERT #1180

Open chenhaoenen opened 2 years ago

chenhaoenen commented 2 years ago

I read your papers about Augmented SBERT and I find them very interesting. I try to reproduce some of the experiment results in your paper. But how do I get exactly the same data set in the paper? image

Meanwhile, I am also eager to reproduce the results of Table 5. Thanks image

nreimers commented 2 years ago

@NThakur20 Can you help here?

thakur-nandan commented 2 years ago

Hi @chenhaoenen,

I will share AugSBERT datasets with exact splits (used in experiments) on this path soon: https://public.ukp.informatik.tu-darmstadt.de/thakur/AugSBERT/datasets/. These datasets with predefined splits will be useful for reproduction for your experiments.

It's difficult to reproduce the results of models. We trained different models for every dataset with variations even at the seed level. Sadly, we can only suggest refer the numbers from the table or reproduce the experiment completely by yourself. Would be happy to help in case you face any issues!

Kind Regards, Nandan

chenhaoenen commented 2 years ago

Thanks for your reply