search

huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
https://huggingface.co/docs/evaluate
Apache License 2.0
1.9k stars 235 forks source link

Can't use the BLEU offline. #565

Open Zhuxing01 opened 3 months ago

Zhuxing01 commented 3 months ago

I download all the metrics from evaluate and save them on the disk. When I want to use BLEU to evaluate my model's outputs, a bug occured. I can't load the BLEU from local dir, the pycharm give me an error like this: ConnectionError: Couldn't reach https://github.com/tensorflow/nmt/raw/master/nmt/scripts/bleu.py (ProxyError(MaxRetryError("HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /tensorflow/nmt/raw/master/nmt/scripts/bleu.py (Caused by ProxyError('Unable to connect to proxy', SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1131)'))))")))

But I swear I loaded it from the local dir.Here's my code: import evaluate metric = evaluate.load(r"E:\evaluate-main\metrics\bleu")

Zhuxing01 commented 3 months ago

This really bothers me. I tried so hard to solve this, but nothing has really worked.

isaacgg commented 1 month ago

I had the same problem.

If you open your file located in E:\evaluate-main\metrics\bleu, you can see a line with an import like this: from .nmt_bleu import compute_bleu # From: https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py

To solve this problem, you have to download this file -> https://github.com/tensorflow/nmt/blob/master/nmt/scripts/bleu.py rename it as nmt_bleu.py and place it in the blue folder.

Finally change the import line to from .nmt_bleu import compute_bleu. It's important to delete the github comment because huggingface looks for these comments using regex to know what external files to download

LYW667 commented 1 month ago

@isaacgg thx, it work~