"Works on your machine"
A Rez superset, on PyPI, for Python 2 and 3, with extended support for Windows, an editable wiki and independent roadmap.
master |
|||
dev |
Rez is a command-line utility for Windows, Linux and MacOS, solving the problem of creating a reproducible environment for your software projects on any machine in any pre-existing environment. It does so by resolving a "request" into a deterministic selection of "packages". Each package is a versioned collection of files with some metadata that you self-host and the resulting "context" is generated on-demand.
$ rez env Python-3.7 PySide2-5.12 six requests
> $ echo Hello reproducible environment!
A small but growing number of companion projects for bleeding- and nerdvegas-rez.
rez-installz
- Native package manager for bleeding-rezrez-localz
- Package localisation from network or cloud storagerez-scoopz
- Install from 500+ system packages for Windows as a Rez packagerez-pipz
- Build and install any PyPI compatible project as a Rez packagerez-cmakez
- Build your projects using CMakerez-yumz
- Install from a selection of 80,000+ RPM packages and countingrez-vcpkgz
- Install any of the 1000+ C++ libraries as a Rez packagerez-conanz
- Install any of the 200+ C++ libraries as a Rez packagerez-npm
- Install any of the 1000+ JavaScript libraries as a Rez packagerez-allzpark
- Visual application launcher and Rez debugging toolrez-for-projects
- A set of example packages for use of Rez (and Allspark) with project and application configurationsrez-performance
- Test the impact of x-number of packages with y-level of complexity in your network environment to make better integration and deployment decisions.rez-releaz
- Secure package releases with Git integrationrez-guiz
- Old-school visual editor of Rez contextsrez-flowchartz
- Visualise package dependencies as a flowchart, useful for debuggingrez-...
Your project here!Here's how to install and use bleeding-rez on your machine.
$ pip install bleeding-rez
$ rez bind --quickstart
$ rez --version
2.33.0
$ rez env
> $ echo Hello World!
Hello World!
The
>
character denotes that you are in a resolved environment, great job!
Now head over to the Quickstart Guide for your first look at what it can do!
How do you test whether your new software runs on another machine? You could run it on another machine. Or you could run it with bleeding-rez.
In order to guarantee that what works on your machine works everywhere, the environment generated by bleeding-rez is "isolated". Akin to what you get out of a Docker container. It contains only the bare essentials provided with a new install of your OS, everything else being provided by your packages.
C:\
__________ __________ __________
| | | | | |
| packageA | | packageB | | packageC |
|__________| |__________| |__________|
| | |
____ v __________ v __________ v ___
| |
| Clean OS |
|____________________________________|
Conversely, nerdvegas/rez sometime inherits the environment of the parent shell, resulting in a mix of variables coming from packages, and some coming from the parent. How does it determine whether to inherit? If a package references a variable, it is overwritten. Otherwise, inherited.
This is one of the core differences between nerdvegas- and bleeding-rez, and is what makes bleeding-rez safe and predictable.
MY_VARIABLE
for granted. Everything runs, until it comes time to make a release. Once released, your users complain that your software doesn't work, and yet on your machine it does!PYTHONHOME
is set to his system Python distribution, rather than the one required
by your package.Unlike nerdvegas/rez, bleeding-rez fully supports packaging of production environments, also known as "profiles".
Some features have been disabled by default. If you encounter any issues, here is how can re-enable them.
rezconfig.py
Apply all of these for full compatibility with nerdvegas/rez
import os
# bleeding-rez does not affect file permissions at all
# as it can cause issues on SMB/CIFS shares
make_package_temporarily_writable = True
# bleeding-rez does not support Rez 1 or below
disable_rez_1_compatibility = False
# nerdvegas/rez inherits all values, except under special circumstances.
# See https://github.com/mottosso/bleeding-rez/issues/70
# Can also be passed interactively, to override whatever is set here.
# $ rez env --inherited # = True
# $ rez env --isolated # = False
inherit_parent_environment = True
# bleeding-rez simplifies the map for Windows
# You can undo this simplification like this.
platform_map = {}
# On Windows, the default shell for bleeding-rez is PowerShell
default_shell = "cmd" if os.name == "nt" else None
rezbuild.py
build system doesn't appear to work without inherit_parent_environment = True
Should I use nerdvegas/rez or bleeding-rez?
If you need to use Rez on Windows with Python 2 and 3, and prefer a simplified installation procedure via PyPI along with the protection of an isolated environment then bleeding-rez is for you.
Why does bleeding-rez exist?
bleeding-rez started off as a fork from which to make PRs to nerdvegas/rez, but eventually started to diverge, now bleeding-rez is a true fork featuring additional safety and cross-platform compatibility, especially with Windows.
Are there any similar projects to bleeding-rez?
Yes, to some extent. Have a look at these.
Project | Scope | Shared Packages | Commercial |
---|---|---|---|
bleeding-rez | ESAP | x | |
rez | ES | x | |
Stash | ESAP | x | x |
be | ESAP | x | |
Ecosystem | E | x | |
add | E | x | |
avalon | EA | x | |
miniconda | S | ||
virtualenv | S | ||
venv | S | ||
pipenv | S | ||
poetry | S | ||
hatch | S | ||
nixpkgs | S | x | |
scoop | S | ||
fips | S | ||
spack | ESA |
How can I get involved?
I'm glad you asked! You're welcome to fork this repository and make a pull-request with your additions. Don't worry too much about the details, automated tests will kick in any time you update your PR. You can also contribute by looking through this README or the wiki for things to improve, and generally just being part of the project. Welcome aboard!
How do I report a bug?
You can do that in the issues section, try and be specific.
What's the advantage of bleeding-rez over Python's virtualenv/venv or Conda?
Aside from those being strictly limited to Python packages and Rez being language agnostic, both venv and Conda couple the installation of packages with their environment, meaning installed packages cannot be shared with other environments. Consider having installed a series of packages, such as PySide2, PyOpenGL, pyglet and other somewhat large libraries. You'll need to spend both time and disk space for each environment you make; essentially once per project.
On the other side of the spectrum, you've got the global installation directory for something like Python, the site-packages
directory. Why not just install everything there, and let whichever project you work on use what it needs? The problem is version. If one of your projects require PySide2-5.9 but another requires 5.13 there isn't much you can do.
Enter Rez. With Rez, you can install every package under the sun, for all platforms at once, and establish environment dynamically as you either run, build or develop your project.
cd my_project
rez env git-2.1 vs-2017 PySide2-5.13 python-3.7 pyglet-1.1b0 PyOpenGL-3.1 -- python setup.py bdist_wheel
That means, less disk space is used and that every package you install is an investment into future projects. At some point, your repository of packages gets so large that a single computer or disk is not enough to contain them all, and that's exactly the kind of situation Rez excels at solving, with a memcached
backend for performance and multi-repository support via the REZ_PACKAGES_PATH
environment variable which works much like PYTHONPATH
for Python.
Some of the largest consumers of packages, Animal Logic, hosts thousands of gigabytes of actively used packages across thousands of computers within a shared environment.
In addition to the high-level comparisons above, these are a few more in-depth comparisons with projects of particular interest.
Ever wanted to run graphical applications, like Maya, in a Docker container? Now bleeding-rez can give you the next best thing. Full compartmentalisation whilst retaining access to system hardware like your GPU.
Like a Docker container, the environment within is completely independent of the environment from which it was entered.
$ export MY_VARIABLE=true
$ docker run -ti --rm centos:7
> $ echo $MY_VARIABLE
$MY_VARIABLE
Like docker run
, environment variables can be passed into a resolved context like this.
$ docker run -e key=value centos:7
$ rez env -e key=value python-3.7