search

Unbabel / COMET

A Neural Framework for MT Evaluation
https://unbabel.github.io/COMET/html/index.html
Apache License 2.0
441 stars 72 forks source link

HF Hub dependency conflicts with latest `tranformers` version. #191

Closed scottfleming closed 6 months ago

scottfleming commented 6 months ago

🐛 Bug

COMET has an explicit dependency for both transformers = "^4.17" and huggingface-hub = "^0.16.0" which will only allow versions up to 0.17.0, but the newest version of transformers (v4.36.1) has its own huggingface-hub dependency which is >=0.19.3,<1.0. This creates dependency conflicts for anyone trying to build a package on COMET and the latest transformers package. Can we do one of the following:

  1. Remove the huggingface-hub dependency from COMET and just keep the transformers dependency given that the former is implicit in the latter;
  2. Use the same style of huggingface-hub dependency that transformers uses (namely "huggingface-hub>=0.19.3,<1.0") to mitigate dependency conflicts;
  3. At very least, update the huggingface-hub dependency to be ^0.19.3.
ricardorei commented 6 months ago

Hi @scottfleming! I like this suggestions but I rather go for 2. Because if for some reason HF decides to decouple the two packages, COMET will stop working (unable to download models).

ricardorei commented 6 months ago

I just released a new version that should solve this dependency issue. Ping me if the error still persists.

PS: I went for option 2

scottfleming commented 5 months ago

Thanks so much, @ricardorei! I'll test this out and let you know if I run into any problems.