Closed ivanleomk closed 7 months ago
and lets not call out cohere directly in the plot just say "closed source embeddings"
I obtained the following results when I ran the script
Model Name | AUC |
---|---|
sentence-transformers/gtr-t5-large | 0.93892 |
embed-multilingual-v3.0 | 0.938904 |
llmrails/ember-v1 | 0.937499 |
infgrad/stella-base-en-v2 | 0.934832 |
BAAI/bge-base-en-v1.5 | 0.931893 |
thenlper/gte-large | 0.93085 |
text-embeddings-ada-v2 | 0.928656 |
Adding a slightly messy benchmarking script to use in order to benchmark cohere against an open source model.
Summary:
This PR adds a benchmarking script to compare the performance of the 'cohere' model against an open-source model, and includes updates to the Readme file and new helper files.
Key points:
embed.py
.Readme.md
file to reflect these changes.finetune.py
for fine-tuning the model.cache.py
andmodels.py
.Added new files
process.py
andvisualise.py
for processing and visualising the results.Generated with :heart: by ellipsis.dev