Support for .env files (for local demo/stack/release files)

Overview

It would be neat if the stackablectl tool would support reading .env files.

The initial use case which brought up the idea of .env files, was the ability to specify if the tool should use local demo files instead of the remote ones located on GitHub. The current workflow looks something like:

Duplicate the demos/demos-v2.yaml file
Make changes to this file, like adding/removing a demo, change URLs to local filepaths, etc.
Provide this "custom" demos file with the --additional-demos-file argument

Initial changes would introduce a simpler workflow. Instead of using the hard coded remote URL to fetch the demos file (see here), the .env file would allow users / developers to explicitly use the local demos file. This would make newly added demos immediately available during development. The workflow would look like this then:

Make changes to the demos/demos-v2.yaml file directly
Add .env file and specify the option introduced by this issue (see below)
Run stackablectl without the --additional-demos-file argument

One benefit this approach offers, is that the .env file must only be created once inside the stackablectl repo. It is of course not checked into Git and each user / developer can choose to use this approach or not.

ENV Option

The ENV option should be prefixed by either STACKABLE or STACKABLECTL. This should be done for all future options (if any). This is done to avoid any conflicts with ENV options from other tools or applications. A few possibilities how this option can look come to mind:

<PREFIX>_USE_LOCAL_DEMOS_FILE: Either true or false, uses the local demos/demos-v2.yaml file. This is not the best option, as we would rely on a relative path and it is not always guaranteed that the user is inside the root folder of the stackablectl repo. On the other hand, the ctl could read the current working directory and join the paths accordingly. Let's discuss this.
<PREFIX>_LOCAL_DEMOS_FILE: Path to the local demos file path, at best an absolute one.
<PREFIX>_ADDITIONAL_DEMOS_FILE: Combine this ENV option with the already present --additional-demos-file CLI argument. An entry in the .env file would behave just like providing the argument by hand. Specifying this option would automate this.

Further discussion

This concept could be extended to also include the stacks and releases files. Using the last naming scheme from above would be the best fit in my opinion.

The proposal can also be extended further by adding the ability to use local paths for the specified remote manifest files. This process can be done explicitly (by providing distinct local and remote URLs / paths). This would introduce a breaking change.

manifests:
  - plainYaml:
      remote: https://raw.githubusercontent.com/stackabletech/stackablectl/main/demos/data-lakehouse-iceberg-trino-spark/serviceaccount.yaml
      local: ./demos/data-lakehouse-iceberg-trino-spark/serviceaccount.yaml
  - plainYaml:
      remote: https://raw.githubusercontent.com/stackabletech/stackablectl/main/demos/data-lakehouse-iceberg-trino-spark/load-test-data.yaml
      local: ./demos/data-lakehouse-iceberg-trino-spark/load-test-data.yaml

The other (implicit) option would be to automatically cut the https://raw.github... part of the URLs and use the remainder as the local path. This would introduce no breaking change.

The proposed features are mostly QoL ones. Introducing support for the .env file can be the basis for further options in the future. Let's hear some feedback from others. I would be open to implement this if approved.

stackabletech / stackablectl