[Enhancement]: Proposal to Isolate the Milvus Conan Recipe from Upstream Changes

[yellow-shine]

Objective:

To isolate the Milvus project from upstream changes, protect our daily builds, and ensure stability by managing C++ third-party packages using Conan, with a focus on leveraging JFrog Cloud and utilizing Conan Recipe Revisions (RREV).

Context and Rationale:

ConanCenter maintains recipes that often require an up-to-date client to benefit from improvements and fixes. These changes can sometimes introduce breaking changes that affect our project. To mitigate this risk, it is essential to isolate our project from these upstream changes and protect our build process.

Key Issues:

1. Breaking Changes in Recipes:

   • Fixes in upstream Conan recipes by the community can alter the behavior of libraries, affecting exported symbols, compiler flags, and build system files.

2. Project Isolation:

   • Relying directly on ConanCenter can introduce instability due to frequent updates and changes.
   • To ensure stability, it is recommended to host our own validated recipes and manage the versions we use in production.

Proposed Solution:

1. Hosting Validated Recipes:

   • Utilize JFrog Cloud to host all recipes required for our project.
   • Validate recipes in our project environment before pushing them to our JFrog Cloud repository.
   • Configure our project to consume packages exclusively from our JFrog Cloud instance.

2. Using Recipe Revisions (RREV):

   • Enable and use Conan Recipe Revisions (RREV) to lock each recipe revision.
   • Instead of using generic package references (e.g., fmt/9.1.0), specify the exact revision (e.g., fmt/9.1.0#c93359fba9fd21359d8db6f875d8a233).
   • This approach ensures that we use a specific, validated version of each recipe, protecting our build from upstream changes.

3. Implementation Steps:

   • Set Up JFrog Cloud Repository:
   • Use the cloud-provided solution at https://milvus01.jfrog.io.
   • Configure our Conan clients to use this repository as the primary remote.
   • Validate and Host Recipes:
   • Validate each recipe in our project environment.
   • Push validated recipes to our JFrog Cloud repository.
   • Lock Recipe Revisions:
   • Enable recipe revisions in Conan 1.x following the activation instructions.
   • Update our package references to include the specific revision hashes.

Benefits:

• Stability: Ensures that our project is isolated from upstream changes and potential disruptions.
• Control: Provides granular control over the versions of third-party packages used in our builds.
• Consistency: Maintains consistent build environments and reduces the risk of build failures due to upstream changes.

Conclusion: In this way, we can significantly enhance the stability and reliability of our build process. This proposal outlines a structured approach to achieving this goal, ensuring that our project remains resilient to external changes and continues to perform optimally.

Reference： https://github.com/conan-io/conan-center-index/blob/master/docs/consuming_recipes.md#isolate-your-project-from-upstream-changes

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.