Missing dependency on evaluate module leads to error.

[sleeper]

The file dsp/modules/finetuning/finetune_hf.py has a dependency on the module evaluate coming from the copy of https://www.philschmid.de/fine-tune-flan-t5#3-fine-tune-and-evaluate-flan-t5, but this dependency is never installed, leading to an error when playing with fine-tuning.

 File "/Users/fros/dev/ia/dspy/emo/train_model.py", line 59, in <module>
    t5_program = tp.compile(EmojiAdviser(), teacher=p_optimized, trainset=examples, **config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/fros/Library/Caches/pypoetry/virtualenvs/emo-fxlg6mj4-py3.11/lib/python3.11/site-packages/dspy/teleprompt/finetune.py", line 157, in compile
    from dsp.modules.finetuning import finetune_hf
  File "/Users/fros/Library/Caches/pypoetry/virtualenvs/emo-fxlg6mj4-py3.11/lib/python3.11/site-packages/dsp/modules/finetuning/__init__.py", line 1, in <module>
    from .finetune_hf import *
  File "/Users/fros/Library/Caches/pypoetry/virtualenvs/emo-fxlg6mj4-py3.11/lib/python3.11/site-packages/dsp/modules/finetuning/finetune_hf.py", line 10, in <module>
    import evaluate
ModuleNotFoundError: No module named 'evaluate'

[okhat]

pip install evaluate?

[sleeper]

OK, I had to add: evaluate, torch, transformers, nltk, rouge-score and transformers[torch]. I'm still facing some errors (but seems mainly link to the architecture selection, as it complains on the fact I do not have a CUDA, XPU, NPU .. device).

Shouldn't these dependencies be part of the pyproject.toml file?

[okhat]

We don't want to depend on them since >90% of users don't need BootstrapFinetune yet. But we do need to raise a nice error that says please install them.

[sleeper]

Indeed. And we can also add it to the documentation :)