Assistants API Support

[Phodaie]

What are your thoughts on supporting Assistants API? parallel function calls are one advantage. I feel the API may change over the next few months, so maybe we should wait.

[jxnl]

no goals for assistant, my money is on marvin ai doing a great job

[PaulGilmartin]

I would also love for the Assistants API to be supported

[jxnl]

what is the application you're looking for?

[Phodaie]

Developing full application is much easier with Assistants API as OpenAI does all the state-management for you. The application just need to keep the entity ids: Assiatand_id, Thread_id, run_id. Also all entities support "metadata" that can be used to store more application specific and situational attributes. This is perfect of server-less apps. Also it seems that OpenAI's focus is on this so it make sense for us to put all of our API related effort on it. I think other LLM instance providers will follow OpenAI direction.

I am working on a multi-agent system. Each agent is an Assistant. These Assistants are used in the same Thread. So I can switch Assistant based on the current state of the Thread.

[jxnl]

right but what about instructor would make it easier, would love to see some pseudo-code

[Phodaie]

The current functionalities of Instructor are supper useful. If the same functionality (structure data extraction etc.) is ported to Assistant API, it can be used for extracting structured data etc. in apps that are already based on Assistant API. It can wrap client.beta.threads.run.create() function.

[PaulGilmartin]

@jxnl For me, the applications of instructor to the assistants API would be identical to that of the chat completions API. I have a chatbot application which uses a graph of assistants, which essentially function as a decision tree: we have a collection of assistants used to attempt to classify what a prompt is about and then a final assistant which answers it. For the classification assistants, it's important that their answer is one of a predefined set of answers we supply (essentially a multiple choice). Right now to achieve this we need to add information to the prompt to ask the LLM to do this, something like:

"Choose one of the options below. Include no other text in your answer".

This proves to be unreliable, especially for models below gpt4 - often the LLM adds extra detail, breaking the expected output structure.

For this type of classification problem, which is I imagine a common use case for the new assistants API, having a library which enforces structured output would be amazing!

[Phodaie]

I think chat api will be deprecated in the next few months as the name implies it was initially intended for a narrow use cases. Instructor is an elegant minimal framework as compared to Marvin, LangChain ... and is suitable for developers who want to build the rest of the functionality themselves. Many of the use cases are not obvious now but will come up over time.

Simple implementation could be at Assistant creation/update point and at run creation time by adding it as tool and instruction.

[jxnl]

😭 this is so much responsibility. Open to some sketch prs!

[covitof]

I have played around with assistants and have modified instructor locally to have some things (e.g. something similar to from_response) work with the assistants framework. I'd be happy to discuss/contribute if needed!

[PaulGilmartin]

@covitof I'd be interested in seeing that if you're able to show us