Open BryceGattis opened 6 months ago
Attention: Patch coverage is 62.96296%
with 20 lines
in your changes are missing coverage. Please review.
:exclamation: No coverage uploaded for pull request base (
main@9c8de33
). Click here to learn what that means.
Files | Patch % | Lines |
---|---|---|
src/rez/artifact_repository.py | 52.94% | 16 Missing :warning: |
src/rez/package_cache.py | 71.42% | 1 Missing and 1 partial :warning: |
src/rez/rezconfig.py | 0.00% | 2 Missing :warning: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Thought for potential backlog item: Can we use a different data validation library or perhaps update our vendored schema
package? The output you get out of it currently is borderline unreadable to a human. For example here's a snippet of a traceback I got while working on this PR:
File "D:\Code\rez\venv\lib\site-packages\rez\vendor\schema\schema.py", line 235, in validate
raise SchemaError([None] + x.autos, [e] + x.errors)
rez.vendor.schema.schema.SchemaError: Or(<class 'rez.utils.sourcecode.SourceCode'>, Schema({Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.util
s.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})})) did not validate 'None'
'None' should be instance of {Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional
('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})}
@BryceGattis I agree the errors are awful. I have started to look at replacing our schema stuff with something more modern. But it's a difficult task because the schemas leak into our public API is in some places, the rez internals (the core ones) are all very tightly coupled to schemas and it is also very performance sensitive. It is probably positive to swap our schema library with something else without introducing breaking changes, but I'll be a tough task.
This PR picks up from the work that work that @ttrently started in PR #1453.
Requirements
This implementation requires some additional packages that have not been included in rez.vendor.
These can be installed via python -m pip install PKG --target path\to\rez\Lib\site-packages.
Setup
For testing I setup Mongo and Mongo Express docker containers locally by running the following commands:
docker run --detach --name mongo_rez --network rez -p 27017:27017 mongo:7.0.5
docker run --detach --name mongo_rez_express --network rez -e ME_CONFIG_MONGODB_SERVER=mongo_rez -p 8081:8081 mongo-express:1.0.2-20
I ran the Mongo Express container so I could easily see the contents of the DB in the browser.
How To Use
In order to use you will need an existing S3 bucket and a MongoDB instance with a named database and a collection called "packages".
In your
rezconfig.py
you will need to set the following:Please note that the above Mongo URI will not directly be usable by PyMongo in a Python Console.
Steps
rez build --install -p mongo@localhost:27017/db_name --clean
to build a package and put its definition in the MongoDB. TODO: Running this seems to upload to DB now butinstall_variant
doesn't fully work yet.Changes
FUTURE
The following changes are not handled by this PR, but should be kept in mind for the future.
rez-release
do we need to makerelease_packages_path
a list so we can deploy to potentially multiple package repositories at once?TODO:
rez/vendor
folder, but maybe that's a good first thing to do?rez release
behavior. Does this push to both to package repository and artifact repository? What does this do if there are multiple of either of those?