turbot / steampipe

Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
https://steampipe.io
GNU Affero General Public License v3.0
6.83k stars 262 forks source link

Workspace options do not seem to apply when running a Flowpipe query step #4101

Closed cbruno10 closed 7 months ago

cbruno10 commented 7 months ago

Describe the bug If I have the following workspace settings:

cbruno@M1P testrunner % cat ~/.steampipe/config/workspaces.spc
workspace "default" {
  cache = false
  cache_ttl = 1
  #cache_ttl = 300

  options "query" {
    # multi     = false      # true, false
    # output    = "table"    # json, csv, table, line
    # header    = false      # true, false
    # separator = ","        # any single char
    timing = true # true, false
  }

  options "check" {
    # multi     = false      # true, false
    # output    = "table"    # json, csv, table, line
    # header    = false      # true, false
    # separator = ","        # any single char
    timing = true # true, false
  }
}

With a Flowpipe pipeline that runs:

  step "query" "list_sqs_queues" {
    connection_string = "postgres://steampipe@localhost:9193/steampipe"

    sql = <<-EOQ
      select
        *
      from
        ${param.conn_name}.aws_sqs_queue
      limit 50
    EOQ

    # Ignore errors so other queries can finish
    error {
      ignore = true
    }
  }

When I run this pipeline, the first result takes 1-2 seconds, and then every subsequent query takes 200-300ms, which makes it seem like caching is not disabled.

Even if I set STEAMPIPE_WORKSPACE=default, the cache = false setting does not seem to take effect.

Steampipe version (steampipe -v) v0.21.4

To reproduce See above

Expected behavior Workspace settings should be applied

Additional context Add any other context about the problem here.

cbruno10 commented 7 months ago

Closing this issue as workspace options only work with Steampipe directly, not with other SQL clients like Flowpipe.