AKissMail / MIRAGEXR_AI_service-backend

0 stars 0 forks source link

rag-model-opmzation #66

Closed AKissMail closed 4 months ago

AKissMail commented 4 months ago

This pull request includes a series of updates and refactorings and bugfixes aimed at improving the quality of responses, the overall code structure, configuration handling, and error management. The key changes include:

Refactor RAG Model Handling and Update Configurations:
    Introduced a factory pattern for managing RAG models.
    Configurations are now dynamically loaded.
    Replaced old JSON config files with updated versions that align with the refactored model handling scheme.
    Fixed issues with file paths in README not resolving correctly.

Refactor Code Structure and Import Paths:
    Restructured 'speak' and 'listen' modules by moving files into a 'models' subdirectory for better organization.
    Adjusted import paths accordingly across various files.

Refactor and Reorganize Configuration Code, Update JSON Filenames:
    Updated JSON filenames in the config/think directory to be more descriptive.
    Refined request handling strategy based on configurations.
    Added interpretable comments and removed unnecessary code parts for better readability and maintenance.

Update Tests, Add CORS Configurations, and Detail README:
    Made tests more comprehensive and isolated.
    Included Cross-Origin Resource Sharing (CORS) configurations in settings.py.
    Added a critical tip in README.md about setting up the CORS_ORIGIN_WHITELIST.

Update Error Messages and Validation in Multiple Files:
    Enhanced error messages in 'handleData.py' and 'views.py' for better context.
    Switched configuration name to 'jaccard' in 'test_views.py'.
    Added 'validate_keys' to configuration check in 'rag_manager.py'.
    Updated 'views.py' under 'think' to return HTTP 400 BAD REQUEST when required keys are missing.
    Added 'django_extensions' to settings.py.

These updates are aimed at improving the maintainability, clarity, and functionality of the codebase. Detailed changes are listed in the individual commits.

Testing: Due to the new model, it can be that the automated test fails. That is due to the embedding model that is not included in the repository. I verified that all tests pass and the application functions as expected. To replicate that, the model cc.no.300.bin has to be downloaded and added in the root folder.