turbot / flowpipe

Flowpipe is a cloud scripting engine. Automation and workflow to connect your clouds to the people, systems and data that matters.
https://flowpipe.io
GNU Affero General Public License v3.0
383 stars 16 forks source link

In `pipeline` steps, feeding a string to the pipeline attribute causes a crash #178

Closed Joeturbot closed 10 months ago

Joeturbot commented 1 year ago

Putting in a pipeline reference as a double quoted string causes the crash below. Changing it back to a regular HCL reference solved the crash.

Causes Crash:

  step "pipeline" "convert_volumes" {
    description = "Convert a list of EBS volumes to the specified volume type."
    for_each    = step.query.get_ebs_volumes.rows
    pipeline    = "aws.pipeline.update_ebs_type_conversion"
    args        = {
      region            = param.aws_region
      access_key_id     = var.aws_access_key_id
      secret_access_key = var.aws_secret_access_key
      volume_type       = param.volume_type
      volume_id         = each.value.volume_id
      iops              = param.iops
    }
  }

Corrected:

  step "pipeline" "convert_volumes" {
    description = "Convert a list of EBS volumes to the specified volume type."
    for_each    = step.query.get_ebs_volumes.rows
    pipeline    = aws.pipeline.update_ebs_type_conversion
    args        = {
      region            = param.aws_region
      access_key_id     = var.aws_access_key_id
      secret_access_key = var.aws_secret_access_key
      volume_type       = param.volume_type
      volume_id         = each.value.volume_id
      iops              = param.iops
    }
  }
2023-10-18T18:09:12.227+0800    ERROR   Recovered from panic    {"error": null}
2023-10-18T18:09:12.227+0800    ERROR   Handler returned error  {"error": "Internal Error: panic occurred: \"value is not a collection\", stacktrace: 
goroutine 403 [running]:
runtime/debug.Stack()
    /opt/hostedtoolcache/go/1.21.1/x64/src/runtime/debug/stack.go:24 +0x5e
github.com/turbot/flowpipe/internal/service/es.(*ESService).Start.PanicRecovererMiddleware.func8.1.1()
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/service/es/middleware/recoverer.go:35 +0x125
panic({0x16c0880?, 0x1eaabb0?})
    /opt/hostedtoolcache/go/1.21.1/x64/src/runtime/panic.go:914 +0x21f
github.com/zclconf/go-cty/cty.Value.LengthInt({{{0x1ec3dc8?, 0xc000012d3f?}}, {0x16c0880?, 0xc000a0ca60?}})
    /home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_ops.go:1162 +0x1e6
github.com/zclconf/go-cty/cty.Value.AsValueMap({{{0x1ec3dc8?, 0xc000012d3f?}}, {0x16c0880?, 0xc000a0ca60?}})
    /home/runner/go/pkg/mod/github.com/zclconf/go-cty@v1.14.0/cty/value_ops.go:1449 +0x71
github.com/turbot/pipe-fittings/modconfig.(*PipelineStepPipeline).GetInputs(0xc001133e40, 0xc000cffd40?)
    /home/runner/work/flowpipe/flowpipe/pipe-fittings/modconfig/flowpipe_pipeline_step.go:1921 +0x1e8
github.com/turbot/flowpipe/internal/es/handler.PipelinePlanned.Handle({0xc000849490?}, {0x1ec39d0?, 0xc000cfe150}, {0x167afc0?, 0xc000cfe780?})
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/es/handler/pipeline_planned.go:261 +0x24eb
github.com/ThreeDotsLabs/watermill/components/cqrs.EventProcessor.routerHandlerFunc.func1(0xc000b70a20)
    /home/runner/go/pkg/mod/github.com/!three!dots!labs/watermill@v1.2.0/components/cqrs/event_processor.go:152 +0x3ff
github.com/ThreeDotsLabs/watermill/message.(*Router).AddNoPublisherHandler.func1(0xc000332000?)
    /home/runner/go/pkg/mod/github.com/!three!dots!labs/watermill@v1.2.0/message/router.go:330 +0x17
github.com/turbot/flowpipe/internal/service/es/middleware.(*PlannerControl).Middleware-fm.(*PlannerControl).Middleware.func1(0xc000b70a20)
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/service/es/middleware/planner.go:76 +0x1c8
github.com/turbot/flowpipe/internal/service/es.(*ESService).Start.LogEventMiddlewareWithContext.func9.1(0xc000b70a20)
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/service/es/middleware/logger.go:60 +0x828
github.com/turbot/flowpipe/internal/service/es/middleware.Retry.Middleware-fm.Retry.Middleware.func1(0x0?)
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/service/es/middleware/retry.go:41 +0x1c
github.com/turbot/flowpipe/internal/service/es.(*ESService).Start.PanicRecovererMiddleware.func8.1(0xc000848500?)
    /home/runner/work/flowpipe/flowpipe/flowpipe/internal/service/es/middleware/recoverer.go:48 +0x9c
github.com/ThreeDotsLabs/watermill/message.(*handler).handleMessage(0xc000cefe10, 0xc000b70a20, 0xc000154fe0)
    /home/runner/go/pkg/mod/github.com/!three!dots!labs/watermill@v1.2.0/message/router.go:759 +0x16d
created by github.com/ThreeDotsLabs/watermill/message.(*handler).run in goroutine 116
    /home/runner/go/pkg/mod/github.com/!three!dots!labs/watermill@v1.2.0/message/router.go:604 +0x2c5
"}
github-actions[bot] commented 11 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 10 months ago

This issue was closed because it has been stalled for 90 days with no activity.