GH action not failing - Githubissues

JeannedArk commented 1 year ago

Hi,

first of all thanks for the action and the great tool.

The GH action is not failing based on the severityThreshold. For example:

name: Kubescape scanning for misconfigurations
on: [push, pull_request]
jobs:
  kubescape:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: kubescape/github-action@main
      continue-on-error: true
      with:
        format: sarif
        outputFile: results
        files: "kubernetes/**.yaml"
        severityThreshold: low

I am setting up the tool and there are multiple high severity issues in my project. The according log output is:

kubescape scan   kubernetes/**.yaml   --severity-threshold low --format sarif --format-version v2 --output results 
{"level":"info","ts":"2023-04-07T05:59:30Z","msg":"Kubescape scanner starting"}
{"level":"warn","ts":"2023-04-07T05:59:31Z","msg":"current version 'v2.0.183' is not updated to the latest release: 'v2.2.6'"}
{"level":"info","ts":"2023-04-07T05:59:35Z","msg":"Downloading/Loading policy definitions"}
{"level":"info","ts":"2023-04-07T05:59:35Z","msg":"Downloaded/Loaded policy"}
{"level":"info","ts":"2023-04-07T05:59:35Z","msg":"Accessing local objects"}
{"level":"warn","ts":"2023-04-07T05:59:35Z","msg":"git scan skipped","error":"failed to get commit information for file: kubernetes/base/deployment.yaml"}
{"level":"info","ts":"2023-04-07T05:59:35Z","msg":"Done accessing local objects"}
{"level":"info","ts":"2023-04-07T05:59:35Z","msg":"Scanning GitLocal"}
{"level":"info","ts":"2023-04-07T05:59:36Z","msg":"Done scanning GitLocal"}

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Controls: 36 (Failed: 16, Excluded: 0, Skipped: 0)
Failed Resources by Severity: Critical — 0, High — 8, Medium — 19, Low — 6

+----------+--------------------------------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
| SEVERITY |                                        CONTROL NAME                                        | FAILED RESOURCES | EXCLUDED RESOURCES | ALL RESOURCES | % RISK-SCORE |
+----------+--------------------------------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
| High     | Resources memory limit and request                                                         |        2         |         0          |       2       |     100%     |
| High     | Resource limits                                                                            |        2         |         0          |       2       |     100%     |
| High     | Resources CPU limit and request                                                            |        2         |         0          |       2       |     100%     |
| High     | CIS-5.7.3 Apply Security Context to Your Pods and Containers                               |        2         |         0          |       2       |     100%     |
| Medium   | Non-root containers                                                                        |        2         |         0          |       2       |     100%     |
+----------+--------------------------------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
|          |                                      RESOURCE SUMMARY                                      |        3         |         0          |       3       |    45.03%    |
+----------+--------------------------------------------------------------------------------------------+------------------+--------------------+---------------+--------------+
FRAMEWORKS: ArmoBest (risk: 42.72), cis-v1.23-t1.0.1 (risk: 66.67), cis-eks-t1.2.0 (risk: 65.38), NSA (risk: 40.51), MITRE (risk: 0.00), DevOpsBest (risk: 51.22), AllControls (risk: 39.24)

{"level":"info","ts":"2023-04-07T05:59:37Z","msg":"Scan results saved","filename":"results.sarif"}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scan results have not been submitted: run kubescape with the '--account' flag
For more details: https://hub.armosec.io/docs/installing-kubescape?utm_campaign=Submit&utm_medium=CLI&utm_source=GitHub
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Run with '--verbose'/'-v' flag for detailed resources view

{"level":"fatal","ts":"2023-04-07T05:59:37Z","msg":"result exceeds severity threshold","set severity threshold":"low"}

As you can see from the logs the threshold is passed to kubescape and the last line confirms result exceeds severity threshold. However, the action is not failing.

dwertent commented 1 year ago

Hi @JeannedArk .

Notice, you are running the actions with continue-on-error: true. This means, that also if the step fails, GitHub actions will continue to the next step without failing.

I advise you to remove the line or set the value to false. e.g.

name: Kubescape scanning for misconfigurations
on: [push, pull_request]
jobs:
  kubescape:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: kubescape/github-action@main
      continue-on-error: false  # GitHub actions should fail in case kubescape exists with a none 0 code
      with:
        format: sarif
        outputFile: results
        files: "kubernetes/**.yaml"
        severityThreshold: low

JeannedArk commented 1 year ago

Hi @dwertent ,

Oh my. I totally missed that. Thank you!

kubescape / github-action

GH action not failing #36