Open XuanRen4470 opened 8 months ago
@XuanRen4470
Yes, please directly download them and load them by json.
You can refer to the evaluator_by_json.py
for the loading code.
Hi Minghao, Thanks for the great work! Could you please share some results and examples on how to evaluate existing models on your benchmark like how to get the results for GPT-3.5? So that it would be easy for others to perform an apple-to-apple comparison on the benchmark.
I tried to use huggingface to download the dataset
but it gave me the error
I also tried to download the json file directly to my local machine, but when reading the files, the input format is like json format instead of a text string.
For example, this is the first input instruction.
"\nGenerate an API request in the format of [ApiName(key1='value1', key2='value2', ...)] based on the previous dialogue context.\nThe current time is 2039-03-09 18:56:09 Wednesday.\nInput: \nUser: User's utterence\nAI: AI's response\n\nExpected output:\nAPI-Request: [ApiName(key1='value1', key2='value2', ...)]\n\nAPI descriptions:\n"
How do i load the dataset properly?
Thank you for your help