ShishirPatil / gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
https://gorilla.cs.berkeley.edu/
Apache License 2.0
11.52k stars 1.01k forks source link

[BFCL] bugs in function def _multi_threaded_inference(self, test_case, include_input_log: bool, include_state_log: bool): #753

Closed pikepokenew closed 2 weeks ago

pikepokenew commented 2 weeks ago

Describe the issue I encountered an error while running bfcl generate. The error occurred in the file gorilla/berkeley-function-call-leaderboard/bfcl/model_handler/oss_model/base_oss_handler.py in the function def _multi_threaded_inference(self, test_case, include_input_log: bool, include_state_log: bool):.

The line model_responses, metadata = self.inference_single_turn_prompting(test_case, include_input_log, include_state_log) caused a runtime error. I discovered that the function inference_single_turn_prompting only accepts the parameters test_case and include_input_log. However, the code additionally passes include_state_log, which leads to the runtime error. When I removed include_state_log, the code ran successfully.

ID datapoint

  1. Datapoint / Model Handler permalink:
  2. Issue:
  3. Gorilla repo commit #:

What is the issue The function inference_single_turn_prompting does not accept include_state_log as a parameter, causing a runtime error when it is passed. Proposed Changes

{ 'previous_datapoint':[], 'updated_datapoint':[] }

Additional context Add any other context about the problem here.

HuanzhiMao commented 2 weeks ago

Thanks for the issue @pikepokenew and good catch!