search

dev-sec / cis-dil-benchmark

CIS Distribution Independent Linux Benchmark - InSpec Profile
Apache License 2.0
149 stars 92 forks source link

False Positives when checking all rules #144

Open bryaend opened 1 year ago

bryaend commented 1 year ago

https://github.com/dev-sec/cis-dil-benchmark/blob/e1dc70a80b65d2cbe10874af4bf01071e3979fbe/controls/3_5_firewall_configuration.rb#L52-L78

When this section is run, I am getting hundreds of failures, one for every rule that doesn't match the filter:

-P INPUT DROP
is expected to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/0.00076s
  Got 2 failures:

  1) Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }

       expected "-P INPUT DROP" to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       Diff:
       @@ -1 +1 @@
       -/(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       +"-P INPUT DROP"
     # cis-dil-benchmark/controls/3_5_firewall_configuration.rb:158:in `block (5 levels) in load_with_context'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `find_all'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `add_resource'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:311:in `get_check_example'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:319:in `block in register_rule'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `flat_map'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `register_rule'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:137:in `block in load'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:135:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:135:in `load'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:163:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/cli.rb:366:in `exec'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/base_cli.rb:35:in `start'

  2) Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }

       expected "-P INPUT DROP" to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       Diff:
       @@ -1 +1 @@
       -/(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       +"-P INPUT DROP"
     # cis-dil-benchmark/controls/3_5_firewall_configuration.rb:158:in `block (5 levels) in load_with_context'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner_rspec.rb:97:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:193:in `run_tests'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:164:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/cli.rb:366:in `exec'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/base_cli.rb:35:in `start'

This should be checking to see if the rule exists in the list, not checking every rule in the list against the matcher.

bryaend commented 1 year ago

Upon checking the documentation, this is how describe.one is expected to work in the case that no check meets the criteria to pass the check, however, it creates a lot of clutter for what is essentially only one failure.

bryaend commented 1 year ago

Also, as a final note, the failures are still present despite the check passing:

-A INPUT -i lo -j ACCEPT
is expected to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/