Benchmark to run all included named controls only once

[thheinen]

Is your feature request related to a problem? Please describe. I want to use this mod to scan a multitude of AWS accounts and post-process the results with my custom tools.

Currently, I would have to run all benchmarks (as I would not assume to have all included queries inside a specific one) or do some scripting around that. Running all benchmarks creates a deeply nested file (JSON in my case) which is hard to parse.

Describe the solution you'd like I would like a "benchmark" grouping all the controls, resulting in coverage of included controls and a shallow report (ideally 1 layer, no nesting). I do not need any tags or docs for my task, as this is supplied on another level.

Ideally, steampipe check benchmark.aws_all_controls --export account.json or something like that

Describe alternatives you've considered

• Executed all benchmarks and did some serious JSONPath wrangling (maintenance issue)
• Checking if the conformance_pack can be used standalone (did not find a way)
• Thinking about if I should script creating a temporary benchmark on top of checked-out code (feels dirty)

Additional context This is part of a multi-tool aggregation of security posture, which enriches the results with more metadata and expert feedback.

[cbruno10]

@tecracer-theinen If I understand correctly, is your main request around producing a smaller/more focused output? Do any of the other output formats work well for where you're exporting the data to, or is JSON the best form?

@judell I know you have done some experimentation with local mods in terms of grouping benchmarks/controls from a mod into a custom benchmark structure. Do you think something like that could work well here?

[johnsmyth]

@tecracer-theinen As you mentioned, you could create a benchmark for this with a script. The introspection tables make this fairly simple, something like:

controls=$(steampipe query "select resource_name from steampipe_control" --output csv --header=false)

echo benchmark \"all_controls\" {
echo  title         = \"All AWS Compliance Controls\"
echo  children = [

for c in $controls; do
  echo control.$c,
done

echo   ]
echo }

[johnsmyth]

@tecracer-theinen Another option might be to create a custom output format? https://steampipe.io/docs/develop/writing-control-output-templates

[judell]

@tecracer-theinen perhaps also of interest: if you run all controls once you will wind up running some queries more than once since controls belonging to different benchmarks may share a common underlying query.

Here's a way to deduplicate:

with controls as (
  select
    (regexp_matches(source_definition, 'query\.([^\.]+)'))[1] as query,
    resource_name as control
  from
    steampipe_control
  order by
    query
)
select distinct on (query)
  'steampipe check control.' || control as check_command
from 
  controls

Without the distinct on (query) you get 467 commands, with it you get 245.

[thheinen]

Hey, thanks for those swift replies. I already had a script for this but the duplicated syntax made everything much smoother.

My request is more or less about having some way of executing all queries in a compliance mod without shell scripting etc. If that's conceptually awkward or defeats its purpose, that is totally fine. I just rather ask for features in the official code base than doing workarounds :-)

[johnsmyth]

Not to hijack, but may be better / simpler to use the steampipe_reference table instead of parsing the source_definition:

 select distinct on (r.reference_to)
    c.resource_name
  from steampipe_control as c 
  left join steampipe_reference as r 
    on concat('control.',c.resource_name) = r.reference_from
  where 
    reference_to like 'query.%'

[cbruno10]

Hi @tecracer-theinen , we currently don't have any plans to introduce a way to run all unique controls only once. I believe the solutions that @judell and @johnsmyth have provided should help with your use case though. If you have any other questions, please let us know in this issue, or in a new one. Thanks!