Using Microsoft Guidance and/or Langchain

[ADTC]

Have you explored using Guidance? It can guarantee the JSON structure. I'm thinking we could use curie model or a cheaper model even, if we use Guidance to generate the exact JSON.

There's also another project called Langchain which makes it easier to do a few things when it comes to integrating with an LLM. So far I discovered that you don't need to parse the PDF into text yourself, and you can specify -1 as the max_tokens value which automatically determines the exact max tokens needed. All these are baked into Langchain.

(At this moment, there's no clarity on whether the two can be used together. Probably not yet.)

[hxu296]

Hi there. I previously did some explorations to let davinci guide curie to accelerate resume paring. A "custom chain" in Langchain's definition, but it's not implemented using Langchain. The control flow is:

1. davinci model parses raw resume text into a list of experience names without details (takes 3-5 seconds).
2. create a new prompt for the curie model using parsed names. Then curie tries to complete full experience details (takes 3 seconds, around 3x speed up compared to davinci).
3. check whether curie has successfully parsed all experiences. If curie's token depleted or stopped early, use davinci model to complete curie's work (takes around 3 seconds if triggered).
4. reverse eliminate duplicated resume bullets, as sometimes curie will repeat the same bullets across experiences. This is purely a hard-coded makeup step for curie. In pure davinci's chain this isn't a issue (takes milliseconds).

Some of my takeaways from these explorations:

1. using curie will accelerate the parsing 2-3 times, making the wait time much more tolerable.
2. curie itself isn't smart enough to do the parsing from raw resume text. It needs informational guidance from a more powerful model to be effective.
3. even with the guidance (not the library, word overloaded lol), curie's output is more messy and requires more extensive post-processing to look reasonable.
4. surprisingly, this chain can parse resume across languages. I tried English, Germain, and Mandarin resume, and they are all parsed decently.
5. cheaper OpenAI chat completion (turbo) model can be used to replace davinci with minimal development overhead. Simply put prompt as user message with empty system message. Turbo is a bit faster than davinci and a lot slower than curie.

About Langchain, my experience is that Langchain's strength is its integrations with other LLM libraries / services like PromptLayer (prompt A/B test and monitoring service) and FAISS (document embedding library). For a simple service like resume parsing, it may not be immensely helpful. However, I am not a Langchain power user. I only used it once in my side project, so more investigations may be needed to be sure.

[hxu296]

BTW guaranteed JSON from Guidance sounds super useful. I will look into that 👍