Engine code for an OpenAI assistant that likes to talk about AI, written using Typescript, Node.js, & the Azure stack. GPT-3 backed with a store of AI Canon documents.
Generate N (100s) of questions that a developer who is learning to apply LLM technology to build new features in their application might ask.
Run each question against the document DB - assess coverage (how may questions do we get a hit) and relevance (for those where we hit, what is the best % match of our document against the question)
Generate follow up question via LLM prompt.
Ask the LLM if the question looks like one that a developer who is learning to apply LLM technology to build new features in their application might ask. = yes/no. Measure hit rate of yes.
In Python.
Example: