FAILED tests/test_benchmark/test_benchmark_integration_with_datasets.py::test_benchmark_sentence_transformer[model0-BrazilianToxicTweetsClassification] - KeyError: "Column homophobia not in the dataset. Current columns in the dataset: ['text', 'label']"
FAILED tests/test_benchmark/test_benchmark_integration_with_datasets.py::test_benchmark_sentence_transformer[model0-task1] - FileNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
I suggest changing the "integration" tasks in tests to tasks with datasets from the MTEB organization, as it would allow us better control over them
When running tests, I encountered an error with
BrazilianToxicTweetsClassification
.In #1300 runner logs:
And in #1323 runner logs:
I suggest changing the "integration" tasks in tests to tasks with datasets from the MTEB organization, as it would allow us better control over them