An end-to-end sample of RAG showcasing development, evaluation, experimentation, and deployment using Promptflow, search products like CosmosDB, PostgresSQL, and Azure AI Search
Added a notebook generate_benchmark_qa.ipynb to preprocessing for generation a number of questions and answers based on text in chunks. This notebook works with more recent version of Azure openai 1.6.1. It does not require additional frameworks such as langchain, to derisk the langchain update intricacies. It should work with previous version of Azure openai (<1.0.0) also, that was used for other purposes in this repo.
There is a prompt template presented that allows using filter parameters. and context texts based on chunks of data. It can be further iterated. Note: Azure OpenAI function call is deprecated in more recent versions, and we want to use tools feature once we have a newer version deployed. (I tested tools with 12-01-2023 preview version, and it doesn't seem to work. We can revisit this later.)
Please run the notebook, and check that each cell works. The notebook shows 10 questions, but more can be generated. Experimentation might be required for further refinement. This is extension of the notebook in rag-e2e-samples, and parsing question/answers and saving to csv is added. Here we also use pagenumber in the filters to concatenate chunks and use them as context (in other case chunks are used directly).
sdk-ai
commented
9 months ago
Added a notebook
generate_benchmark_qa.ipynbto preprocessing for generation a number of questions and answers based on text in chunks. This notebook works with more recent version of Azure openai 1.6.1. It does not require additional frameworks such as langchain, to derisk the langchain update intricacies. It should work with previous version of Azure openai (<1.0.0) also, that was used for other purposes in this repo.There is a prompt template presented that allows using filter parameters. and context texts based on chunks of data. It can be further iterated. Note: Azure OpenAI function call is deprecated in more recent versions, and we want to use tools feature once we have a newer version deployed. (I tested tools with 12-01-2023 preview version, and it doesn't seem to work. We can revisit this later.)
Please run the notebook, and check that each cell works. The notebook shows 10 questions, but more can be generated. Experimentation might be required for further refinement. This is extension of the notebook in rag-e2e-samples, and parsing question/answers and saving to csv is added. Here we also use pagenumber in the filters to concatenate chunks and use them as context (in other case chunks are used directly).