-
I have two questions:
1. How do you compute the reported BLEU scores? I see the imports of BLEU in several models but BLEU is never added to the `self.metrics` collection of any of them.
2. When…
-
Why do I only get a CIDEr score of 0.065 on Flickr30k for bliva_vicuna7b, even if multiplied by 10 it's only 0.65? Could you tell me what might have gone wrong in this process?? Thanks.
{"test": {"Bl…
-
Hi,
I was working on a basic example in sentence_bleu. It gives me an error about keywords like this:
```
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Fraction.__new__(…
-
@haruhi-sudo
"info": {
"task": "QuestAnswer3Docs",
"llm": "{'model_name': 'qwen7b', 'temperature': 0.1, 'max_new_tokens': 1280, 'top_p': 0.9, 'top_k': 5}"
},
"overall":…
-
In the Bleu function,What does test_text and real_text mean? Does test_text refer to 'save/test_file.txt'? And what does real_text refer to?' save/oracle.txt' or 'save/generator.txt'? Thanks!
-
-
`In [1]: from sacrebleu.metrics import BLEU, CHRF`
` ...: refs = [["请关掉灯光。"]]`
` ...: sys = ["请关闭灯光。"]`
`In [2]: bleu = BLEU(trg_lang="zh")`
`In [3]: bleu.corpus_score(sys, refs)`
`Out[3]: BL…
-
- Paper: [BLEU: a Method for Automatic Evaluation of Machine Translation](https://aclanthology.org/P02-1040.pdf)
- Blog: https://jaketae.github.io/study/bleu/
- Wiki: https://en.wikipedia.org/wiki/B…
-
I download all the metrics from evaluate and save them on the disk. When I want to use BLEU to evaluate my model's outputs, a bug occured. I can't load the BLEU from local dir, the pycharm give me an…
-
De crème