turbot / flowpipe

Flowpipe is a cloud scripting engine. Automation and workflow to connect your clouds to the people, systems and data that matters.
https://flowpipe.io
GNU Affero General Public License v3.0
361 stars 13 forks source link

Pipeline Indexing #837

Closed graza-io closed 2 days ago

graza-io commented 5 months ago

Context

Was discussing observations as per Issue 835 where I can pass a final name part as a Pipeline index (for a dependency mod).

step "transform" "pl_name" {
  value = "list_s3_buckets" // this could exist in both my and a mod dependency
}

step "pipeline" "action" {
  pipeline = pipeline[step.transform.pl_name.value]
  args = {
    region = param.region
    cred   = param.cred
  }
}

Since the indexing appears to be done on a short-name (example below) - this could cause complications where we have pipeline naming conflicts in a parent mod and one of it's dependencies.

{
  "approval": {
    "description": null,
    "documentation": null,
    "max_concurrency": null,
    "name": "approval.pipeline.approval",
    "short_name": "",
    "tags": null,
    "title": null,
    "unqualified_name": "pipeline.approval"
  },
  "associate_iam_ec2_instance_profile": {
    "description": "Associates an IAM instance profile with a running or stopped instance. You cannot associate more than one IAM instance profile with an instance.",
    "documentation": null,
    "max_concurrency": null,
    "name": "aws.pipeline.associate_iam_ec2_instance_profile",
    "short_name": "",
    "tags": null,
    "title": "Associate IAM to EC2 Instance Profile",
    "unqualified_name": "pipeline.associate_iam_ec2_instance_profile"
  },
  ... (omitted for brevity)

This could be problematic if as you can see above, I have a dependency mod with a pipeline approval if I also have approval pipeline and then fire:

step "transform" "name" {
  value = "approval"
}

step "pipeline" "action" {
  pipeline = pipeline[step.transform.name.value]
}

This seemingly would take the last one added to the map.

Additional Notes

Should pipelines in mod dependencies be indexed this way?

Not really sure on the expected behaviour here; just that while this provides a nice quick way to reference pipelines - could be problematic.

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 2 days ago

This issue was closed because it has been stalled for 90 days with no activity.