Azure / gpt-rag-ingestion

MIT License
64 stars 55 forks source link

Refactored components to work asynchronously and updated openai sdk version #83

Closed mateoperezrivera closed 3 months ago

mateoperezrivera commented 3 months ago

This pull request focuses on refactoring the codebase to use asynchronous operations, which should improve the performance and responsiveness of the application. The most important changes include converting synchronous Azure SDK calls to their asynchronous counterparts, updating the document analysis and chunking functions to be asynchronous, and refactoring the TextEmbedder and ChunkEmbeddingHelper classes to support asynchronous operations.

Asynchronous Azure SDK Calls:

Asynchronous Document Analysis:

Asynchronous Chunking:

Refactoring Helper Classes:

These changes aim to improve the efficiency of document processing by leveraging asynchronous I/O operations