Missing import for SadeemQuestionRetrieval

[PrithivirajDamodaran]

Under retrieval tasks for Arabic language SadeemQuestionRetrieval is not recognised by MTEB. Neither lists nor runs.

Please advise.

[KennethEnevoldsen]

Added a fix in #1146. It should now run using:

from __future__ import annotations

import mteb

tasks = mteb.get_tasks(tasks=["SadeemQuestionRetrieval"])
tasks[0].load_data()

model = mteb.get_model("sentence-transformers/paraphrase-MiniLM-L6-v2")

eval= mteb.MTEB(tasks=tasks)
res = eval.run(model)

[PrithivirajDamodaran]

~Thanks, Simple pip upgrade would work or should I install from source?~

Installed from source, still I get this

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[<ipython-input-10-6d7f5f513191>](https://localhost:8080/#) in <cell line: 4>()
      2 import mteb
      3 
----> 4 tasks = mteb.get_tasks(tasks=["SadeemQuestionRetrieval"])
      5 tasks[0].load_data()
      6 

2 frames
[/usr/local/lib/python3.10/dist-packages/mteb/overview.py](https://localhost:8080/#) in get_task(task_name, languages, script)
    237                 f"KeyError: '{task_name}' not found and no similar keys were found."
    238             )
--> 239         raise KeyError(suggestion)
    240     return TASKS_REGISTRY[task_name]().filter_languages(languages, script)

KeyError: "KeyError: 'SadeemQuestionRetrieval' not found. Did you mean: VideoRetrieval?"

[KennethEnevoldsen]

It was in PR #1146, so it was merged yesterday, so you might have tested before it was merged. The latest version of MTEB should work now.