This PR adds support for userspace operations to Gridware - that is, using Gridware as a non-root user.
All Gridware operations are supported:
Installing Gridware packages from binary or source
Exporting depots to a directory
Installing depots
Creating and purging user depots
Installing Gridware packages from user repos (including user clones of the volatile repo)
On login, configuration files and a user depot are created under ~/gridware. By default, packages installed to user depots take priority over system depots. This can be managed via $MODULEPATH / module use / module unuse to reorder the depots' modules directories.
Additional packages from Perl CPAN, Python PyPI etc are installed to a user's home directory, allowing cluster-wide use.
In order to deal with installation of distro package dependencies for non-root users, a new whitelisting system has been created. Administrators can edit /opt/gridware/etc/whitelist.yml, containing the following three lists:
:users: users listed here are allowed to install any distro package.
:packages: the distro packages listed here may be installed by any user.
:repos: distro packages that are dependencies of packages in Gridware repos at the filesystem paths listed here may be installed by any user.
If a user tries to install a Gridware package that has distro dependencies not meeting these criteria, then a request is created to the system administrator(s). These can be viewed with the new alces gridware requests list command, and approved or rejected with alces gridware requests install. The latter supports the --yes option to automatically approve all pending requests; otherwise the admin is prompted for each request. Approved packages are added to the whitelist so that they can be automatically installed on compute nodes when required.
If an admin has configured an email address $cw_GRIDWARE_admin_email in $cw_ROOT/etc/gridware.rc, then they receive an email notification when a new request is generated.
If a user has configured an email address :user_email in ~/gridware/etc/gridware.yml, then they receive an email notification when a request is approved.
Both of these actions can be customised by editing the distro-deps-notify and the package-install-notify scripts installed to /opt/clusterware/libexec/share. (See clusterware-services PR.)
In order to ensure that existing Gridware binary packages use the new distro dependency management system, we detect old-style dependency scripts within the packages and automatically replace them with newly-generated ones that defer to alces gridware dependencies.
This PR adds support for userspace operations to Gridware - that is, using Gridware as a non-root user.
All Gridware operations are supported:
volatilerepo)On login, configuration files and a user depot are created under
~/gridware. By default, packages installed to user depots take priority over system depots. This can be managed via$MODULEPATH/module use/module unuseto reorder the depots' modules directories.Additional packages from Perl CPAN, Python PyPI etc are installed to a user's home directory, allowing cluster-wide use.
In order to deal with installation of distro package dependencies for non-root users, a new whitelisting system has been created. Administrators can edit
/opt/gridware/etc/whitelist.yml, containing the following three lists::users: users listed here are allowed to install any distro package.:packages: the distro packages listed here may be installed by any user.:repos: distro packages that are dependencies of packages in Gridware repos at the filesystem paths listed here may be installed by any user.If a user tries to install a Gridware package that has distro dependencies not meeting these criteria, then a request is created to the system administrator(s). These can be viewed with the new
alces gridware requests listcommand, and approved or rejected withalces gridware requests install. The latter supports the--yesoption to automatically approve all pending requests; otherwise the admin is prompted for each request. Approved packages are added to the whitelist so that they can be automatically installed on compute nodes when required.If an admin has configured an email address
$cw_GRIDWARE_admin_emailin$cw_ROOT/etc/gridware.rc, then they receive an email notification when a new request is generated.If a user has configured an email address
:user_emailin~/gridware/etc/gridware.yml, then they receive an email notification when a request is approved.Both of these actions can be customised by editing the
distro-deps-notifyand thepackage-install-notifyscripts installed to/opt/clusterware/libexec/share. (Seeclusterware-servicesPR.)In order to ensure that existing Gridware binary packages use the new distro dependency management system, we detect old-style dependency scripts within the packages and automatically replace them with newly-generated ones that defer to
alces gridware dependencies.Related PRs:
clusterware-services: https://github.com/alces-software/clusterware-services/pull/52clusterware-handlers: https://github.com/alces-software/clusterware-handlers/pull/79