search

AcademySoftwareFoundation / rez

An integrated package configuration, build and deployment system for software
https://rez.readthedocs.io
Apache License 2.0
942 stars 335 forks source link

Added CustomPackageOrder #1709

Open isohedronpipeline opened 6 months ago

isohedronpipeline commented 6 months ago

This PR carries on the work by @pmolodo (https://github.com/AcademySoftwareFoundation/rez/pull/355) to add a custom package order. This is essentially a generalization of the version_split package order, which provides better functionality for splitting versions within specified ranges, instead of only allowing a single split, as version_split does. This makes the package orderer useful outside of the extremely narrow band of intended uses that the version_split package order was intended to solve.

CustomPackageOrder

This adds support to explicitly specify the way versions should resolve, while still providing flexibility for ranges. For example consider the following package houdini:

versions = [
    "18.5.499",
    "18.5.625",
    "19.0.350",
    "19.0.720",
    "19.5.632",
    "19.5.679",
    "20.0.413",
    "20.0.528",
]

And consider the following optional plugin plugin that has variants (all latest versions of Houdini):

variants = [
    ["houdini-18.5.625"],
    ["houdini-19.0.720"],
    ["houdini-19.5.679"],
    ["houdini-20.0.528"],
]

But let's consider that the latest version of Houdini is not well tested and we do not know if our pipeline will behave properly with these latest versions. On top of that, consider that we want houdini-19.5 to be the default.
Examples:

This is hard to achieve without explicit presets, which makes preference of second order dependencies very difficult to resolve without taking advantage of the graph evaluation of rez. More information here: https://docs.google.com/document/d/1zy3ydSpHWgvatFS--L2PK7AwwOaowr1sdI9B_2eNtFc/edit?usp=sharing

We can now achieve that with the CustomPackageOrder:

package_orderers = [
    {
       "type": "custom",
       "packages": {
           "python": ["19.5.632", "19.5", "20.0.413"],
       }
    }
]

In the above package orderer, we see that we prefer 19.5.632 if no other version constraints are put on it. If 19.5.632 is not suitable (as in the case of plugin not having a requirement match) then we prefer the latest version that fits the VersionRange of 19.5. If that is not suitable, then we prefer 20.0.413 to cause houdini-20 to resolve to a less-than-latest version of Houdini 20.

Lastly, the CustomPackageOrder may take an argument to determine how versions within those version ranges are sorted, e.g. with pypa sorting.

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 76.19048% with 15 lines in your changes missing coverage. Please review.

Project coverage is 58.33%. Comparing base (3c75a19) to head (0af2985). Report is 47 commits behind head on main.

Files with missing lines Patch % Lines
src/rez/package_order.py 76.19% 9 Missing and 6 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1709 +/- ## ========================================== + Coverage 58.25% 58.33% +0.08% ========================================== Files 126 126 Lines 17157 17220 +63 Branches 3504 3520 +16 ========================================== + Hits 9995 10046 +51 - Misses 6496 6503 +7 - Partials 666 671 +5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.