Improve configuration handling of parameters, envars, config file

[gmatev]

Context: The current scheme is not super clear and has caused some concerns with users who are struggling to change key parameters that are anchored to envars or config file.

Proposal: - Use a hierarchical model of overrides along the lines of what is available for the AWS CLI - https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html. In order of precedence:

• Explicit flags
• Envars
• Config file
• (For storage creds only)
  • Credential provider
  • Instance role

For the above to be effective we also need to:

• Expand the set of parameters available as flags - this should cover everything available in ENVARS and config, in addition to other things that may be flags only. Needs a deeper audit but all evars (including what is users for sensitive items) should have flags
• Available envars can stay as is. Add envar to control config file location.
• Config file - this may need to be expanded to capture mode flags. At the very least all flags and envars that are required for init and/or connect today.

Config file and secrets:

• Users can specify a secret in config file (e.g. app client secret, or storage creds), but Corso will never implicitly persist such values on behalf of users

Corso persisting config file automatically:

• If no file exists, any non-sensitive parameters that are effectively used (including defaults) for init or connect are persisted in the corso config file location. Should include user facing output that the config file is created
• If file exists, parameters follow the precedence rules above, but there are some open questions around what should be overwritten.

Open questions:

• Should an overwrite be allowed on init if config exists? If yes, this needs to include requirement to change all parameters that are related to a given repo - all app config settings, bucket settings, passphrase. Any sensitive settings need to be deleted - seems complicated
• Connect is similar to init if the settings are actually different

One option is to show an error if the config file exists (with conflicting params) and allow an overwrite based on something like --overwrite-config. For simplicity this is treated as just deleting the file and persisting (after validation) all effective settings when init or connect is executed successfully

[neha-Gupta1]

@gmatev need a little more explanation - For storage creds only-

• Credential provider
• Instance role Are we thinking of introducing IAM here? And have roles based accesses?

[neha-Gupta1]

Tracked in - https://github.com/alcionai/corso/issues/3586

[neha-Gupta1]

Do we want to update config file values in case of repo Init command?

[neha-Gupta1]

The config vars in - env, flags and config file is done