search

YuanGongND / ltu

Code, Dataset, and Pretrained Models for Audio and Speech Large Language Model "Listen, Think, and Understand".
367 stars 33 forks source link

Eval_metrics #29

Closed joemzhao closed 5 months ago

joemzhao commented 5 months ago

Hi Yuan, thanks a lot for the nice work.

It seems the evaluation scripts for computing cider/spice scores are missing. For example: """ from eval_metrics import evaluate_metrics """ (https://github.com/YuanGongND/ltu/blob/main/src/ltu/eval/eval_audiocaps.py#L14)

could you share about this? Thank you very much

YuanGongND commented 5 months ago

hi there,

Thanks for pointing this out.

The script depends on this package: https://github.com/audio-captioning/caption-evaluation-tools (not our work, but metrics script used by DCASE).

We share our script just to show how we postprocess the prediction. The evaluation part is standard.

-Yuan

joemzhao commented 5 months ago

Understood -- thanks for the help!

YuanGongND commented 5 months ago

Let me know if there's any problem running the script. I think you should be able to run it if you put our script in the same dir of that package.

-Yuan

joemzhao commented 5 months ago

Hi Yuan -- thanks for the kind help! Yes, I can run the scripts to obtain the scores properly.