search

alces-software / adminware

A sandbox CLI for running commands remotely across nodes
1 stars 0 forks source link

Prototype adminware command generation #133

Closed WilliamMcCumstie closed 5 years ago

WilliamMcCumstie commented 5 years ago

Thanks to the assistance of @bobwhitelock, there is a dedicated function for generating commands: appliance_cli.command_generation.generate_commands. It was originally designed as a wrapper cli that is configured using a dictionary. In order for it to be useful, the config directory structure needs to be compiled into a single dict.

https://github.com/alces-software/imageware/blob/4efa15d9f243ba18aa6dc250c6f71d073ef6dd2a/controller/cli/src/fly_wrapper_config.py#L45-L73

A new Config.hashify_all method takes the configs from a glob and a template command hash. The template command is duplicated for each Config (at the hashes leaves). The template command also supports functions, lambda, and other callable objects as dictionary values; with the config as the input.

All of this has been wrapped in a Config.command decorator which can be used in the CLI. Some adjusted to the option parsing was required. The --node and --group flags must be placed after the tool name now. The nodes option unpacking has also been placed in a decorator.

Basic caching of the Config objects has also been added. This PR addresses parts of #126 and #114.

DavidMarchant commented 5 years ago

I'll merge this through now as discussed Two things: First is the definition of CONFIG DIR in models/config https://github.com/alces-software/adminware/pull/133/files#diff-9dec05cb851210ad467cd073ec726c7cR14 - shouldn't this be in the top level config module with the rest of the path definitions for consistency & to be as DRY as poss.? https://github.com/alces-software/adminware/blob/refactor/command-generation/src/config.py#L19-L21

The other is the use of glob - https://github.com/alces-software/adminware/pull/133/files#diff-9dec05cb851210ad467cd073ec726c7cR36 - how does this work around the issue that was our original motivation for abandoning glob in favour of manual recursion to begin with? The case in which there is a large file tree sibling to a config.yaml that may itself contain a file called config.yaml. That being possibly a) extremely computationally intensive & b) flooding the cli with deep namespace suggestions. Have you changed stance on this?