asynchronous operations & updating openai version.

[gbecerra1982]

This pull request focuses on refactoring the chunk_documents_docint.py, chunk_documents_raw.py, chunk_metadata_helper.py, and text_embedder.py files to use asynchronous operations. The most important changes include converting synchronous functions to asynchronous ones, updating Azure SDK imports to their asynchronous counterparts, and ensuring proper session and resource management.

Asynchronous Refactoring:

• chunker/chunk_documents_docint.py: Converted functions such as get_secret, analyze_document_rest, get_chunk, and chunk_document to asynchronous versions. Updated Azure SDK imports to use asynchronous modules and replaced synchronous HTTP requests with aiohttp requests. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

• chunker/chunk_documents_raw.py: Updated the chunk_document function to be asynchronous and modified the generate_chunks_with_embedding call to use await. [1] [2] [3] [4]

• chunker/chunk_metadata_helper.py: Updated the ChunkEmbeddingHelper class to include asynchronous methods and added a class method for asynchronous initialization.

• embedder/text_embedder.py: Refactored the TextEmbedder class to support asynchronous operations, including asynchronous initialization and embedding methods. [1] [2] [3]

[placerda]

Refactored old chunkers into multiple chunks to be more modular and extensible, so changes in chunk_documents_docint.py, chunk_documents_raw.py, chunk_metadata_helper.py, and text_embedder.py wont be effective.