Closed vinayak1 closed 1 year ago
Hi, these are the Pytorch and Cuda versions we use: Nvidia RTX 3090Ti GPUs Transformers 4.2.1 PyTorch 1.10 CUDA Version: 11.4
Results can vary based on different hardware/software.
Thanks!
Are you able to release the checkpoints for sup-PromCSE-RoBERTa-base? I cannot reproduce results due to hardware differences I presume. Thanks
Hi, the checkpoint for sup-PromCSE-RoBERTa-base has been uploaded to Huggingface (https://huggingface.co/YuxinJiang/sup-promcse-roberta-base). I hope this helps you :)
Hi, I'm trying to reproduce the results for the repo for the supervised training and am using the following environment:
V100 (p3.2xlage) Transformers 4.2.1 PyTorch 1.12 CUDA Version: 11.3
When using these settings I get the following results:
Bert-base-uncased ------ test ------ | STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness | Avg. | | 74.46 | 83.81 | 79.25 | 86.02 | 81.07 | 84.00 | 80.65 | 81.32 |
Roberta-base ------ test ------ | STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness | Avg. | | 76.23 | 86.07 | 80.71 | 86.72 | 83.32 | 85.95 | 80.24 | 82.75 |
It looks like the results are slightly off compared to the reported ones, can you please share the Pytorch and Cuda versions used for training?
Thanks!