Create a Generator that would generate an Open API spec from a passed in Ruby class (Improve Tool creation for Assistants)

patterns-ai-core / langchainrb

Build LLM-powered applications in Ruby

https://rubydoc.info/gems/langchainrb

MIT License

1.36k stars 192 forks source link

Create a Generator that would generate an Open API spec from a passed in Ruby class (Improve Tool creation for Assistants) #541

Closed andreibondarev closed 3 weeks ago

andreibondarev commented 7 months ago

Description

We'd like to improve the experience of creating Tools (classes that inherit from Langchain::Tool::Base) for Assistants to use. Open API spec is difficult to compose and we'd like to see if an LLM itself can generate the JSON spec given a Ruby class.

Tasks

Create a .yaml prompt that instructs an LLM to generate a JSON Open API spec (example: database.json) and accepts a Ruby class.
Create a Generator class (under /utils? or maybe new directory /generators?) where this functionality will live.
This Generator class will output a .json file with the generator Open API spec.

mattlindsey commented 6 months ago

Google added an enable_automatic_function_calling option to their python library that doesn't seem to requre a spec definition. Maybe do something like this: https://github.com/google-gemini/cookbook/blob/main/examples/Agents_Function_Calling_Barista_Bot.ipynb

They say:

In the Python SDK you can pass functions directly to the model constructor, where the SDK will inspect the type signatures and docstrings to define the tools. For this reason it's important that you correctly type each of the parameters, give the functions sensible names and detailed docstrings.

andreibondarev commented 6 months ago

@mattlindsey Nice find! Unfortunately, unlike Python, Ruby does not have built-in type annotations.

andreibondarev commented 6 months ago

@mattlindsey I attached a branch where I'm working on an approach: generate-method-annotations

mattlindsey commented 6 months ago

@andreibondarev Looks great. I can test it for you when ready. A further enhancement could be to make the annotation files optional by looking at the YARD annotations, or for tools that use RBS gem annotations.

Olgagr commented 2 months ago

I'm a little bit confused how should I define tool class for an assistant. I watched the ecommerce assistant presentation: https://www.loom.com/share/83aa4fd8dccb492aad4ca95da40ed0b2?sid=12b11c98-c6b7-4d98-8be8-1f35a2b3861a. However, the linked repository changed a lot. There is no JSON function definition any more. Is the repository state connected with changes introduced by this thread?

andreibondarev commented 2 months ago

I'm a little bit confused how should I define tool class for an assistant. I watched the ecommerce assistant presentation: https://www.loom.com/share/83aa4fd8dccb492aad4ca95da40ed0b2?sid=12b11c98-c6b7-4d98-8be8-1f35a2b3861a. However, the linked repository changed a lot. There is no JSON function definition any more. Is the repository state connected with changes introduced by this thread?

Hi @Olgagr, here's the short answer:

Your Tool class does not need to inherit from Langchain::Tool::Base anymore.
Your Tool class needs to be extended with extend Langchain::ToolDefinition.
Your Tool class needs to declare all methods you'd like to expose using the define_function ... methods. See example: https://github.com/patterns-ai-core/langchainrb/blob/main/lib/langchain/tool/news_retriever.rb#L14-L40

I'll try recording an updated video today to showcase the new syntax. Btw -- I updated the demo with the gem updates.

Olgagr commented 2 months ago

Awesome! Thank you @andreibondarev !

andreibondarev commented 3 weeks ago

This is no longer needed since we got rid of the static .json files and @dghirardo built a beautiful interface here: https://github.com/patterns-ai-core/langchainrb/pull/708