Support for AsyncOpenAI

[Maxwe11y]

I tried to use the customized llm with strict_json. However, I found that the current version does not support AsyncOpenAI and this module is important for massive generation. I recommend the developer to add support for AsyncOpenAI.

[tanchongmin]

Perhaps you can try importing the AsyncOpenAI llm as a separate llm (See 5. Integrating with your own LLM)

res = strict_json(system_prompt = 'You are a classifier',
                    user_prompt = 'It is a beautiful and sunny day',
                    output_format = {'Sentiment': 'Type of Sentiment',
                                    'Adjectives': 'List of adjectives',
                                    'Words': 'Number of words'},
                                     llm = llm) # set this to your own LLM

print(res)

Did the AsyncOpenAI version fail with this?

[Maxwe11y]

Perhaps you can try importing the AsyncOpenAI llm as a separate llm (See 5. Integrating with your own LLM)

res = strict_json(system_prompt = 'You are a classifier',
                    user_prompt = 'It is a beautiful and sunny day',
                    output_format = {'Sentiment': 'Type of Sentiment',
                                    'Adjectives': 'List of adjectives',
                                    'Words': 'Number of words'},
                                     llm = llm) # set this to your own LLM

print(res)

Did the AsyncOpenAI version fail with this?

Yeah, AsyncOpenAI fail with this. I always got a {} response.

[Maxwe11y]

I have another question. If use the open json mode, can I manually copy the code of strict_json function and use it to parse the obtained response as sometimes the response may break the json.loads() rules. Thank you!

[tanchongmin]

Hi Maxwe11, in this case, it may be better to implement your own version of the chat() function in the strictjson base.py. Feel free to modify as needed.

Also, refer to https://github.com/tanchongmin/strictjson/issues/5#event-12442700047 for how to solve the async issue.

There needs to be a custom wrapper over strict_json to get it to work async. I will work on implementing this into the official strictjson repo soon!

[md598]

Ya, Maxwe11y you could try the approach I have used just don't specify the custom groq llm/api like I am. Don't pass that parameter at all. Please note as tanchongmin stated - this will multi-thread it and each thread will be locked while the API itself is running. This allows you to run some calls in parallel but isn't truly async at this time (you can't kick off 100 API calls... only 1/thread in the pool and then each thread will be locked until completion or timeout).

[tanchongmin]

Refer to StrictJSON v5.0.0. There is AsyncFunction and strict_json_async