Project Specific Global Argument Overrides

[en0]

It would be useful to be able to override the config-gen utilities global arguments (--library, --store) for a specific project when ran without any arguments set.

Use Case:

Project A is a side project exploring many new features of a new technology. It might turn into something great, it might be a flop. The company is willing to pay for this discovery process but the DevOps team does not want to add more keys to an already large key store.

The developer has one of 2 options:

1. Add the new keys to his local default key store.
2. Create a new key store for the project.

Both of these options are good ideas but they both have negative impact to the developer's workflow.

Analysis of Current Option 1.

Add the new keys to his local default key store.

One of the things config-gen is trying to solve is the complexity of environment setup. Many environments are inherently complicated. As such many development teams create a centralized stack for development resources such as a database or logging endpoints. Although this is not always true it usually happens at some level. For my team, it is common to share large portions if not all of the key store file between the developers or even setup a centralized key store shared by development.

In the case of a local key store file this solution might be an option with a small price of merge complications and sanitization when the developer receives a new copy or shares a copy of the key store. In this case, config-gen is adding overhead to the workflow of development and is not an ideal solution.

Analysis of Current Option 2

Create a new key store for the project.

Another problem config-gen is solving for developers is to reduce the opportunity of mistakes made by manual manipulation of configuration settings. Especially when that manipulation is a consistent development task. Config-gen is supposed to make this easy. The desired workflow is: The developer runs config-gen in his/her project and config-gen takes care of the rest. And certainly this is true for most use cases.

In this case, that will most likely cause an error or possibly configure the project against the wrong resources. Since the developer's workflow for every other project is to run config-gen and press enter, during different points of development, and often those points are separated by long periods of time, it is likely that the developer will, at some point, forget to override the key store.

Solution Profile

In the case of these situations it would make sense to set a project specific override that config-gen checks and sets the global options for any given execution of config-gen, config-gen-store, and config-gen-key. This file could be a global resource that can identify a project and impose overrides or a project specific resource that config-gen searches for to impose overrides. In any case, the overrides should only be applied if the options are not explicitly stated by the user at runtime.

Solution Workflow

Here is the logical workflow when config-gen is ran

1. config-gen parses the command line arguments.
2. config-gen loads the overrides for this project
3. config-gen will merge the command-line arguments with any overrides given priority to command-line options.
4. config-gen will continue normal execution using the identified library and store.

   Post-hoc, ergo, proto-hoc

Using the above situation (Use case), the developer interactions should look like this.

# Overrides exist for --store priv:debug

~/Project/RnD-Project/ $ config-gen
# Config-gen will read the command line arguments: Results None (all defaults)
# Config-gen will read the project overrides and find --store priv:debug
# Config-gen will change the use of store='default' to store='priv:debug'
# Config-gen will open the default library and lookup the priv:debug details
# config-gen will connect to the uri identified in the library for priv:debug
# Config-gen will render the template using keys from the priv:debug store.

~/Project/RnD-Project/ $ config-gen --store qa
# Config-gen will read the command line arguments: Result store='qa'
# Config-gen will read the project overrides and find --store priv:debug
# Config-gen will not set store='priv:debug' due to a explicitly set command-line option
# Config-gen will open the default library and lookup the qa details
# config-gen will connect to the uri identified in the library for qa
# Config-gen will render the template using keys from the qa store.

[cbigler]

I've got a solution for this mostly complete, using a project local file for overrides. I'm running into an issue being able to tell which options were supplied via CLI and which are populated from the defaults in the argparser.

I'm considering removing the defaults from the parser argument definitions and just applying them manually after if they are missing from the final options.

Thoughts?

[en0]

That is an unfortunate thing. I played with argparse for a bit trying to find a way to identify if the value was set or defaulted. I could not come up with anything. I don't have any other ideas.

I am ok with setting the defaults/override based on a None default.

ap.add_argument('--foo')
opts = ap.parse_args(args)
_foo = opts.foo or override('foo') or 'default_foo'

The above example is to clarify logic only. I have no preference as to the actual implementation.

Thank's for grabbing this @cbigler!