search

ShishirPatil / gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
https://gorilla.cs.berkeley.edu/
Apache License 2.0
11.51k stars 1.01k forks source link

[BFCL] Introducing custom handlers #756

Open sangmandu opened 1 week ago

sangmandu commented 1 week ago

Hi there,

I am a current user of bfcl and would like to submit this PR to address an improvement opportunity I've identified while working with the module.

Currently, model handlers are managed through the handler_map.py, which presents some challenges. This managed approach requires code updates from maintainers whenever new models are released. Additionally, users must wait for updates if their desired model isn't yet supported.

While technically proficient users might implement their own solutions for unsupported models, this especially becomes problematic when using bfcl as a submodule. Modifying the original code is not ideal from a maintenance perspective, as it requires additional resources for ongoing maintenance.

Therefore, I'm submitting this PR to enhance the main codebase.

The current build_handler function only allows fetching predefined model handlers. I propose implementing a HandlerLoader that enables users to input custom handlers. This approach minimizes changes to the main codebase while allowing users to manage their custom handlers through just two files: a Python file containing the handler definition and a JSON configuration file.

sangmandu commented 1 week ago

Thanks for the PR @sangmandu ! Could you provide an example BFCL_HANDLER_CONFIG?

@HuanzhiMao

Wow, rapid reply! Thanks. It looks like our timing is a little off, and I've provided an example in the code explanation section.