uber / cadence

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
https://cadenceworkflow.io
MIT License
8.01k stars 777 forks source link

feat: Inquiry on Configuration Version Management and Automation in Cadence #5833

Open cubxxw opened 3 months ago

cubxxw commented 3 months ago

Hello Cadence Team,

I'm currently exploring Cadence for workflow management and have a particular interest in how configurations are managed within the system. My query revolves around two main points:

  1. Configuration Version Management:

    • Does Cadence support or plan to support version management for configurations?
    • I'm looking for a way to manage my configuration changes without relying on Git version control. This is crucial for scenarios where configurations might contain sensitive information or when we prefer to decouple configuration management from our code repository for better modularity and security.
  2. Configuration Automation:

    • Is there any existing or planned feature in Cadence for the automation of configuration management tasks? Specifically, I'm interested in functionalities similar to kubeadm init, such as:
      • Pre-processing and validation checks for configuration files before applying them.
      • Automatic generation and management of configurations based on predefined templates or conditions.
      • Any tools or integrations that facilitate dynamic configuration changes in a production environment without manual intervention.

The ability to manage configurations efficiently and securely is crucial for our use case, especially in a dynamic and scalable environment. It would greatly enhance our operational workflow and security posture if Cadence could support or is planning to support these features.

Thank you for your time and consideration. I look forward to your insights and any recommendations you might have.

demirkayaender commented 3 months ago

Hi!

For 1) We (and as far as I know other open source users too) rely on an internal config management system to manage including versioning our dynamic configs. This is mostly because config management systems exist in every company for all of their products and it is desirable to standardize it company-wide. Having said that, config manager is passed as an argument to cadence services, so if you'd like to build a default one inside Cadence, we can help you get those changes in. Currently given the size of our backlog, this will be lower priority.

Bullet 2 is similar to number 1 too. We would rather like to keep that logic outside of Cadence. Validation can be done via health checks + automated rollbacks, and impact of bad config deployments can be limited by slow/controlled rollouts (e.g. per zone, per region deployments). Leaving this responsibility to an external tool helps us focus on Cadence core and help users adopt the best technology that they prefer for this.