bclavie
commented
4 weeks ago Hey!
I'm not quite sure that this issue is within scope of what we'd like to do here. rerankers is purposefully a very light, thin library so that it can be slotted in anywhere, and building docker images with other packages for various usecases isn't what we want to do for it at all, since it's simple enough for users to do so and only include the packages that they personally care about.
Is there any fundamental issue with the library that makes it difficult for you to build your own Docker images including it? If so, I'd be more than happy to take a look at what's causing the issues!
Issue: Pre-packaging Docker Image for Re-Ranker with Pre-Downloaded Libraries Description: The current setup for the Re-Ranker application involves downloading libraries and packages at runtime when the Docker container is initiated. This approach introduces several challenges, particularly in environments where internet access is restricted or where applications are subject to strict network proxies.
In our deployment environment, all network traffic is routed through proxies, and internet access may be limited or heavily controlled. As a result, the application faces delays, failures, or interruptions when attempting to download required libraries and packages at runtime. This dependency on real-time downloading can lead to instability and unreliability, especially when running the application in environments with such network constraints.
Proposal: To mitigate these issues, the Flash Rank Re-Ranker Docker image should be pre-packaged with all necessary libraries, dependencies, and language model (LLM) packages already downloaded and included. By doing so, the application will be fully self-contained and will not need to access external resources when the container is started. This ensures that the application can run smoothly and reliably, even in environments with strict network controls or limited internet access.
Benefits:
Improved Reliability: The application will no longer be dependent on external network conditions to function correctly. Faster Startup Times: Since all dependencies are pre-downloaded, the container can start up more quickly without waiting for network downloads. Network Independence: The application can run in offline environments or environments with restricted internet access. Consistency: Ensures that the same versions of libraries and packages are used across different environments, reducing the risk of version conflicts or incompatibilities. This approach will significantly enhance the robustness and reliability of the Flash Rank Re-Ranker application in production environments.