search

AlibabaResearch / DAMO-ConvAI

DAMO-ConvAI: The official repository which contains the codebase for Alibaba DAMO Conversational AI.
MIT License
1.08k stars 176 forks source link

About reproducing the results on level 3 evaluation on API-Bank #114

Open zfchenUnique opened 6 months ago

zfchenUnique commented 6 months ago

Hi, Thanks for sharing the excellent benchmark of API-bank! @liminghao1630 While I am trying to run the https://github.com/AlibabaResearch/DAMO-ConvAI/blob/main/api-bank/lv3_evaluator.py, it seems the prediction json file is missing. Could you please help share json prediction file for reference? Moreover, it will be really nice if you can provide some end-to-end evaluation file like https://github.com/AlibabaResearch/DAMO-ConvAI/blob/main/api-bank/evaluator.py for evaluation.

Regards

zfchenUnique commented 6 months ago

Hi, @huybery @liminghao1630 Thanks again for your exciting work! Is there any update on the issue mentioned? Regards,

liminghao1630 commented 6 months ago

@zfchenUnique We will update and inform you as soon as we have prepared it. Thank you for your attention to us.

jing-P commented 3 months ago

so,did you handle this problem？i don't know how to use the lv3_evaluator.py