search

awslabs / seed-farmer

Seed-Farmer is an orchestration tool that works with AWS CodeSeeder and acts as an orchestration tool modeled after GitOps deployments. It has a CommandLine Interface based in Python, leverages modular code deployments defined by declarative manifests, and includes change detection and deployment optimization.
https://seed-farmer.readthedocs.io/en/latest/
Apache License 2.0
45 stars 16 forks source link

[FEATURE] Support Cross-Deployment Metadata Sharing #668

Open a13zen opened 1 month ago

a13zen commented 1 month ago

Is your feature request related to a problem? Please describe. Support metadata export/import across deployments. If users can import/export metadata across deployments, they can have shared resources such as EKS/Airflow/Buckets etc that don't need to be namespace specific in one deployment, and then they can import and use the shared resources in their namespace deployment.

This would also allow reducing the overall Role footprint as users don't need to deploy copies of the shared resources. It would also allow reducing costs while still supporting namespaces.

Describe the solution you'd like

  - name: EKS_CLUSTER_NAME
    valueFrom:
      moduleMetadata:
        deployment: "shared-eks-cluster" <----- NEW
        group: eks
        name: shared-cluster
        key: ClusterName

Dependency management can be completely manual. If the target entry in SSM doesn't exist. FAIL the deployment of the module depending on it (can be checked during validation). If the value in SSM is garbage, or out of date, that is up to the end user to ensure they deploy dependencies in the correct order.

dgraeber commented 1 month ago

@a13zen We will evaluate this request and review our dependency management schemes. Our primary concern here is deleting dependency modules. The concept of a forceDependencyRedeploy will need to be limited in scope to a single deployment, so the manual management is indeed something to be aware of.