mottosso / bleeding-rez

Rez - Reproducible software environments for Windows, Linux and MacOS
GNU Lesser General Public License v3.0
71 stars 10 forks source link

REZ_PLUGIN_PATH #27

Open mottosso opened 5 years ago

mottosso commented 5 years ago

Externalise plug-ins, and enable external plug-ins being written.

mottosso commented 5 years ago

For subcommand-plugins, a naive approach would be to look at REZ_PLUGIN_PATH whenever rez is invoked..

$ rez env python-3

But that would cause common commands such as rez env to grow slower and slower as the number of plug-ins grow and their initialisation take longer.

Instead, only look for subcommand plug-ins when asked for a non-existent subcommand.

$ rez install scoopz
# "install" subcommand not found..
# searching for "install" subcommand plug-in..

That way, performance will still drop, but only for plug-ins themselves. The downside is that this type of plug-in couldn't affect a pre-existing subcommand like env which is probably OK.

mottosso commented 5 years ago

For subcommands, not only do we need to facilitate the commands themselves, but also arguments to existing commands.

$ rez install localz
$ rez env python-3 --localised -- python -c "print('hello local computer!')"

Here, --localised is provided by the third-party localz package.

Like for subcommands themselves, we don't want plug-ins to negatively affect existing commands, and could probably employ the same technique of only searching for a plug-in argument once no argument has been found.

mottosso commented 5 years ago

Feature Advertisement

A plug-in must advertise features added, such that other plug-ins can be alerted to the fact that they exist.

For example, if a plug-in attempts to add rez localise when another plug-in have already added this command, it should fail. The same goes for arguments like rez env --localised.

Installation

A Rez package should provide installation instructions for a given plug-in, such that it may be installed both as a package, and as a plug-in if plug-in support is provided.

For example, rez-localz is both a Rez package and plug-in.

package.py

name = "localz"
version = "1.0.0"

def commands():
  env.PATH.prepend("{root}/bin")

def plugin():
  env.REZ_PLUGIN_PATH.prepend("{root}/python")

Here, on rez build --install or rez install the plugin() function is invoked and operates on the system-wide environment as opposed to the environment of the resulting subshell. On Windows and cmd, that's e.g. setx and on Linux that's e.g. appending to ~/.bashrc. Details need some working out.

Alternatively, a plug-in can have its own definition.

plugin.py

# Optional, defaults to package.py:name
# Only one plug-in of this name may exist on REZ_PLUGIN_PATH
name = "localz"

# Optional, subcommands added by this plug-in
commands = ["localise"]  # i.e. rez localise

# Optional, arguments provided to what other sub-command
arguments = {"rez env": ["--localised"]}

From there, the plug-in manager could handle appending appropriate paths to REZ_PLUGIN_PATH and validate against pre-existing subcommands and arguments.

Some plug-ins could then have only a plugin.py such that they are only Rez plug-ins, and not packages. From there, they could get installed via pip install and added retroactively via Rez.

$ pip install rez-localz
$ rez plugin --install rez-locals  # searching PYTHONPATH per default
mottosso commented 5 years ago

plugin_for

As it happens, the Rez CLI already has a notion of a "plug-in", but it's not the same. Packages are able to advertise themselves as "plug-ins".

$ rez plugin bifrost  # is a plug-in to..
maya-2018

What are our options?

Given that rez plugin is effectively a metadata query of a package, we could potentially deprecate this and make another command such as view more generic..

$ rez view six --attribute commands
env.PYTHONPATH.prepend("{root}/python")
$ rez view bifrost --attribute plugin_for
maya-2018
mottosso commented 5 years ago

Like for Python and https://github.com/pypa/sampleproject, we should provide a suggested hierarchy for external plug-ins written and hosted on e.g. GitHub.

Example

rez-myproject/
  bin/          # (optional) Added to PATH
  build/        # (transient) Created during local build
  python/       # (optional) Added to PYTHONPATH
  package.py    # (mandatory) Standard Rez definition
  setup.py      # (mandatory) Standard PyPI installation script
  README.md     # (mandatory) Describe your project
  LICENCE       # (mandatory) Preferably The Unlicence (i.e. public domain)

Where setup.py is a regular PyPI compatible script for Python-like projects, substituted with CMakeLists.txt for CMake, conanfile.py for Conan or CONTROL for VcPkg and so forth.

The root-level package.py could then be the minimum required content for use with rez install.