Use of Function/Tool with OpenAI

[matias-biatoz]

2023-06-13 OpenAI's announcement of the API's changes allows to pass a function parameter which supposedly improves the LLM interpretation of the task. (Langchain already implemented the necessary changes in 0.199/0.200) Do you see how can this be use to improve Kor data extraction?

[eyurtsev]

I haven't used the new API for extraction yet, so I don't have a sense of quality. Previously gpt-3.5-turbo seemed significantly worse than text-davinci-003.

If anyone is willing to run some experiments, the thing to do is:

Set up a zero and few shot scenario with the new API (with AIMessages that contain the function invocation request payload specified), and see how it performs against text-davinci-003.

Compare with json encoding, and also with a csv encoding. It would be good to confirm that the new chat API does a good job at extracting multiple entities for a long passage of text.

[eyurtsev]

Folks should try this:

https://github.com/hwchase17/langchain/blob/356dc429329f80c57451552cce1b7a5553d4e86b/docs/extras/modules/chains/additional/extraction.ipynb

Please if you try this in your data and see any differences in performance let me know!

[matias-biatoz]

I have been trying out 3 different ways of data extraction.

• Super custom: It works the best but is not scalable
• Kor it works really well but sometimes it hallucinates
• OpenAI works well but not as well as kor. I managed with kor to correctly interprent user's intent in a chat but with OpenAI's function could not. However, it never hallucinated but it sometimes returns '' or '0' when it should be None.

[tomerzcod7]

I also found that if you need to do "semantic extraction" (basically trying to realize what the user actually means) Kor has better results for me at the moment. I think maybe some way of combining Kor with functions to force a structured results (and prevent hallucinations) would get optimal results