feat: Inquiry on Configuration Version Management and Automation in Cadence

[cubxxw]

Hello Cadence Team,

I'm currently exploring Cadence for workflow management and have a particular interest in how configurations are managed within the system. My query revolves around two main points:

1. Configuration Version Management:

   • Does Cadence support or plan to support version management for configurations?
   • I'm looking for a way to manage my configuration changes without relying on Git version control. This is crucial for scenarios where configurations might contain sensitive information or when we prefer to decouple configuration management from our code repository for better modularity and security.

2. Configuration Automation:

   • Is there any existing or planned feature in Cadence for the automation of configuration management tasks? Specifically, I'm interested in functionalities similar to kubeadm init, such as:
     • Pre-processing and validation checks for configuration files before applying them.
     • Automatic generation and management of configurations based on predefined templates or conditions.
     • Any tools or integrations that facilitate dynamic configuration changes in a production environment without manual intervention.

The ability to manage configurations efficiently and securely is crucial for our use case, especially in a dynamic and scalable environment. It would greatly enhance our operational workflow and security posture if Cadence could support or is planning to support these features.

Thank you for your time and consideration. I look forward to your insights and any recommendations you might have.

[demirkayaender]

Hi!

For 1) We (and as far as I know other open source users too) rely on an internal config management system to manage including versioning our dynamic configs. This is mostly because config management systems exist in every company for all of their products and it is desirable to standardize it company-wide. Having said that, config manager is passed as an argument to cadence services, so if you'd like to build a default one inside Cadence, we can help you get those changes in. Currently given the size of our backlog, this will be lower priority.

Bullet 2 is similar to number 1 too. We would rather like to keep that logic outside of Cadence. Validation can be done via health checks + automated rollbacks, and impact of bad config deployments can be limited by slow/controlled rollouts (e.g. per zone, per region deployments). Leaving this responsibility to an external tool helps us focus on Cadence core and help users adopt the best technology that they prefer for this.

[ibarrajo]

I think this idea is very close to the same concept in sys_config in mysql or pg_settings in postgres. To my knowledge they don't support versioning out of the box but you could have a table history log to track changes.

Special design considerations must be given to the hierarchy of the configuration to keep it consistent and avoid the possibility of a cluster being unable to be restarted after a configuration error or depending on the database layer being available.