The "AutoStudentReportLLMs" repo features a complete implementation for automated student report evaluation, utilizing advanced language models (e.g., BERT, Mistral). It encompasses UI code, seamless PDF/Word integration, and detailed documentation on architecture, technologies, and test results.
2
stars
0
forks
source link
Enhance Query Generation for Vector Database Assessment #2
This pull request introduces significant improvements to the generate_queries function within our project. The updated function now meticulously crafts prompts to solicit the AI model for generating precise and structured queries, aimed at querying a vector database. These queries are crucial for assessing the fulfillment of various task criteria.
Key Changes:
Detailed Prompting: The function now constructs a more detailed and instructive prompt. It explicitly asks the AI model to generate questions based on specific grading criteria associated with a given task. The intent is to gauge whether the task's objectives have been achieved.
JSON Formatting: Emphasis has been placed on formatting the output as a JSON object. Each generated question correlates to a particular criterion, ensuring that questions are systematically organized by their relevance to the task's assessment criteria.
Enhanced Output Structure: The expected output format has been clearly defined within the prompt, guiding the model to produce outputs in a {"NAME OF CRITERION": ["QUESTION 1", "QUESTION 2", ...]} format. This structured approach facilitates easier parsing and utilization of the generated questions for database queries.
Error Handling: Improved error handling for JSON decoding, ensuring that any issues encountered during the parsing of model responses are clearly logged, allowing for swift identification and rectification of issues.
File Writing: The function consolidates the generated questions into a single JSON file, generated_queries.json, stored within a specified prompt_directory. This file encapsulates all the questions mapped against their respective criteria, prepared for subsequent processing or analysis.
This pull request introduces significant improvements to the generate_queries function within our project. The updated function now meticulously crafts prompts to solicit the AI model for generating precise and structured queries, aimed at querying a vector database. These queries are crucial for assessing the fulfillment of various task criteria.
Key Changes:
Detailed Prompting: The function now constructs a more detailed and instructive prompt. It explicitly asks the AI model to generate questions based on specific grading criteria associated with a given task. The intent is to gauge whether the task's objectives have been achieved.
JSON Formatting: Emphasis has been placed on formatting the output as a JSON object. Each generated question correlates to a particular criterion, ensuring that questions are systematically organized by their relevance to the task's assessment criteria. Enhanced Output Structure: The expected output format has been clearly defined within the prompt, guiding the model to produce outputs in a {"NAME OF CRITERION": ["QUESTION 1", "QUESTION 2", ...]} format. This structured approach facilitates easier parsing and utilization of the generated questions for database queries.
Error Handling: Improved error handling for JSON decoding, ensuring that any issues encountered during the parsing of model responses are clearly logged, allowing for swift identification and rectification of issues.
File Writing: The function consolidates the generated questions into a single JSON file, generated_queries.json, stored within a specified prompt_directory. This file encapsulates all the questions mapped against their respective criteria, prepared for subsequent processing or analysis.