search

nimblehq / infrastructure-templates

For IaaS and PaaS as codes
MIT License
10 stars 0 forks source link

[#227] Migrate from tfsec to Trivy #262

Closed nvminhtue closed 7 months ago

nvminhtue commented 10 months ago

What happened 👀

Integrate Trivy that will replace the current tfsec

Insight 📝

Proof Of Work 📹

Generate the completed AWS services locally and detect no HIGH or CRITICAL issues. image

The example of a failure check without putting the trivy ignores image

CI will be failed if the Trivy scan found any HIGH or CRITICAL severity image

Nihisil commented 9 months ago

@nvminhtue it looks like something is not working properly.

  1. I have generated new project from your branch
  2. Deleted few trivy:ignore lines there and there

When I'm running trivy locally I'm receiving exit error without any details:

$ trivy config .
2023-12-30T11:40:37.723+0700    INFO    Loaded trivy.yaml
2023-12-30T11:40:37.736+0700    INFO    Misconfiguration scanning is enabled
2023-12-30T11:40:38.522+0700    INFO    Detected config files: 13
exit 1

When trivy run on CI there are no errors at all, CI run is green:

Running Trivy with trivy.yaml config from:  trivy.yaml
2023-12-30T04:37:57.267Z    INFO    Loaded trivy.yaml
2023-12-30T04:37:57.276Z    INFO    "--dependency-tree" only shows the dependents of vulnerable packages. Note that it is the reverse of the usual dependency tree, which shows the packages that depend on the vulnerable package. It supports limited package managers. Please see the document for the detail.
2023-12-30T04:37:57.276Z    WARN    "--dependency-tree" can be used only with "--format table".
[20](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:21)23-12-30T04:37:57.283Z INFO    Need to update DB
2023-12-30T04:37:57.283Z    INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2023-12-30T04:37:57.283Z    INFO    Downloading DB...
32.83 MiB / 42.08 MiB [----------------------------------------------->_____________] 78.02% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [-------------------------------------------------] 100.00% 24.40 MiB p/s 1.9s2023-12-30T04:37:59.724Z    INFO    Vulnerability scanning is enabled
20[23](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:24)-12-30T04:37:59.724Z INFO    Secret scanning is enabled
2023-12-30T04:37:59.7[24](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:25)Z INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-12-30T04:37:59.724Z    INFO    Please see also https://aquasecurity.github.io/trivy/v0.45/docs/scanner/secret/#recommendation for faster secret detection
2023-12-30T04:37:59.740Z    INFO    Number of language-specific files: 0

I expected to see errors locally and in CI for the blocks where I removed the ignore statements. Could you please help me figure out what's wrong with my tests?

nvminhtue commented 9 months ago

@nvminhtue it looks like something is not working properly.

  1. I have generated new project from your branch
  2. Deleted few trivy:ignore lines there and there

When I'm running trivy locally I'm receiving exit error without any details:

$ trivy config .
2023-12-30T11:40:37.723+0700  INFO    Loaded trivy.yaml
2023-12-30T11:40:37.736+0700  INFO    Misconfiguration scanning is enabled
2023-12-30T11:40:38.522+0700  INFO    Detected config files: 13
exit 1

When trivy run on CI there are no errors at all, CI run is green:

Running Trivy with trivy.yaml config from:  trivy.yaml
2023-12-30T04:37:57.267Z  INFO    Loaded trivy.yaml
2023-12-30T04:37:57.276Z  INFO    "--dependency-tree" only shows the dependents of vulnerable packages. Note that it is the reverse of the usual dependency tree, which shows the packages that depend on the vulnerable package. It supports limited package managers. Please see the document for the detail.
2023-12-30T04:37:57.276Z  WARN    "--dependency-tree" can be used only with "--format table".
[20](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:21)23-12-30T04:37:57.283Z   INFO    Need to update DB
2023-12-30T04:37:57.283Z  INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2023-12-30T04:37:57.283Z  INFO    Downloading DB...
32.83 MiB / 42.08 MiB [----------------------------------------------->_____________] 78.02% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 15.43 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 14.44 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [-------------------------------------------------] 100.00% 24.40 MiB p/s 1.9s2023-12-30T04:37:59.724Z  INFO    Vulnerability scanning is enabled
20[23](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:24)-12-30T04:37:59.724Z   INFO    Secret scanning is enabled
2023-12-30T04:37:59.7[24](https://github.com/Nihisil/test-infra/actions/runs/7362083108/job/20040144448#step:6:25)Z   INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-12-30T04:37:59.724Z  INFO    Please see also https://aquasecurity.github.io/trivy/v0.45/docs/scanner/secret/#recommendation for faster secret detection
2023-12-30T04:37:59.740Z  INFO    Number of language-specific files: 0

I expected to see errors locally and in CI for the blocks where I removed the ignore statements. Could you please help me figure out what's wrong with my tests?

@Nihisil, It actually generates the errors into the trivy-output.json file. But it's a good catch, the CI never notices if there is any mis-configurated, updated the trivy config file in d5a52da and f6e6628

Nihisil commented 9 months ago

@nvminhtue thank you. After your fix I can receive error on localhost, but CI is still the green.

Also I have deleted all trivy ignore lines, and it shows me only one error. Do we need to keep ignore lines in that case, or it is misconfiguration and it should show more errors?

And another issue is that it doesn't show where the error is, based on error message it is impossible to find out where the fix is required. Is it possible to adjust it to show file and line number?

local output:

2024-01-03T10:56:32.454+0700    INFO    Loaded trivy.yaml
2024-01-03T10:56:32.462+0700    INFO    Misconfiguration scanning is enabled
2024-01-03T10:56:35.821+0700    INFO    Detected config files: 26

 (terraform)

Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Cluster does not have Deletion Protection enabled
═══════════════════════════════════════════════════════════════════════════════════════════════════════════
Ensure deletion protection is enabled for RDS clusters.

See https://avd.aquasec.com/misconfig/n/a
───────────────────────────────────────────────────────────────────────────────────────────────────────────

exit 1

CI output:

Running Trivy with trivy.yaml config from:  trivy.yaml
2024-01-03T03:57:34.451Z    INFO    Loaded trivy.yaml
2024-01-03T03:57:34.460Z    WARN    "--list-all-pkgs" cannot be used with "--format table". Try "--format json" or other formats.
2024-01-03T03:57:34.460Z    INFO    "--dependency-tree" only shows the dependents of vulnerable packages. Note that it is the reverse of the usual dependency tree, which shows the packages that depend on the vulnerable package. It supports limited package managers. Please see the document for the detail.
[20](https://github.com/Nihisil/test-infra/actions/runs/7393288283/job/20112986384#step:6:21)24-01-03T03:57:34.465Z INFO    Need to update DB
2024-01-03T03:57:34.465Z    INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2024-01-03T03:57:34.465Z    INFO    Downloading DB...
26.84 MiB / 42.08 MiB [-------------------------------------->______________________] 63.78% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [-------------------------------------------------] 100.00% 25.48 MiB p/s 1.9s2024-01-03T03:57:36.788Z    INFO    Vulnerability scanning is enabled
2024-01-03T03:57:36.788Z    INFO    Secret scanning is enabled
20[24](https://github.com/Nihisil/test-infra/actions/runs/7393288283/job/20112986384#step:6:25)-01-03T03:57:36.788Z INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-01-03T03:57:36.788Z    INFO    Please see also https://aquasecurity.github.io/trivy/v0.45/docs/scanner/secret/#recommendation for faster secret detection
2024-01-03T03:57:36.800Z    INFO    Number of language-specific files: 0
nvminhtue commented 9 months ago

@nvminhtue thank you. After your fix I can receive error on localhost, but CI is still the green.

Also I have deleted all trivy ignore lines, and it shows me only one error. Do we need to keep ignore lines in that case, or it is misconfiguration and it should show more errors?

And another issue is that it doesn't show where the error is, based on error message it is impossible to find out where the fix is required. Is it possible to adjust it to show file and line number?

local output:

2024-01-03T10:56:32.454+0700  INFO    Loaded trivy.yaml
2024-01-03T10:56:32.462+0700  INFO    Misconfiguration scanning is enabled
2024-01-03T10:56:35.821+0700  INFO    Detected config files: 26

 (terraform)

Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)
Failures: 1 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Cluster does not have Deletion Protection enabled
═══════════════════════════════════════════════════════════════════════════════════════════════════════════
Ensure deletion protection is enabled for RDS clusters.

See https://avd.aquasec.com/misconfig/n/a
───────────────────────────────────────────────────────────────────────────────────────────────────────────

exit 1

CI output:

Running Trivy with trivy.yaml config from:  trivy.yaml
2024-01-03T03:57:34.451Z  INFO    Loaded trivy.yaml
2024-01-03T03:57:34.460Z  WARN    "--list-all-pkgs" cannot be used with "--format table". Try "--format json" or other formats.
2024-01-03T03:57:34.460Z  INFO    "--dependency-tree" only shows the dependents of vulnerable packages. Note that it is the reverse of the usual dependency tree, which shows the packages that depend on the vulnerable package. It supports limited package managers. Please see the document for the detail.
[20](https://github.com/Nihisil/test-infra/actions/runs/7393288283/job/20112986384#step:6:21)24-01-03T03:57:34.465Z   INFO    Need to update DB
2024-01-03T03:57:34.465Z  INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2024-01-03T03:57:34.465Z  INFO    Downloading DB...
26.84 MiB / 42.08 MiB [-------------------------------------->______________________] 63.78% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [----------------------------------------------------------->] 100.00% ? p/s ?42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 25.38 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [---------------------------------------------->] 100.00% 23.74 MiB p/s ETA 0s42.08 MiB / 42.08 MiB [-------------------------------------------------] 100.00% 25.48 MiB p/s 1.9s2024-01-03T03:57:36.788Z  INFO    Vulnerability scanning is enabled
2024-01-03T03:57:36.788Z  INFO    Secret scanning is enabled
20[24](https://github.com/Nihisil/test-infra/actions/runs/7393288283/job/20112986384#step:6:25)-01-03T03:57:36.788Z   INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-01-03T03:57:36.788Z  INFO    Please see also https://aquasecurity.github.io/trivy/v0.45/docs/scanner/secret/#recommendation for faster secret detection
2024-01-03T03:57:36.800Z  INFO    Number of language-specific files: 0

Not sure how did you try on your local, everything works fine on my end image

To make sure we are on the same page, have you generated a local template and removed the trivy:ignore then?

The CI won't run since I couldn't find the way to unbundle the template by script, hence no tf files to run through, I believe we can do that on a separated story.

Nihisil commented 9 months ago

To make sure we are on the same page, have you generated a local template and removed the trivy:ignore then?

Yep, this is what I did.

Can you please check it on this branch: https://github.com/Nihisil/test-infra/tree/test-trivy

It shows an error that I provided above, and I'm not sure where the issue is.

Nihisil commented 9 months ago

The CI won't run since I couldn't find the way to unbundle the template by script, hence no tf files to run through, I believe we can do that on a separated story.

We need to ensure it runs on CI, or we should remove it from the linting step of GH workflow job and create a ticket to add it later. Otherwise, there might be confusion where developers might think that Trivy validation is already working in CI

nvminhtue commented 9 months ago

The CI won't run since I couldn't find the way to unbundle the template by script, hence no tf files to run through, I believe we can do that on a separated story.

We need to ensure it runs on CI, or we should remove it from the linting step of GH workflow job and create a ticket to add it later. Otherwise, there might be confusion where developers might think that Trivy validation is already working in CI

I created the story here, will work on that first and apply the change to this one, which can ensure that trivy is successfully integrated.

Nihisil commented 9 months ago

I'm sorry that I wasn't clear earlier. Trivy isn't working with the generated project, neither on localhost nor on CI for me. This is most important part.

As for the ticket that you created, we have it already: https://github.com/nimblehq/infrastructure-templates/issues/181

nvminhtue commented 9 months ago

I'm sorry that I wasn't clear earlier. Trivy isn't working with the generated project, neither on localhost nor on CI for me. This is most important part.

As for the ticket that you created, we have it already: #181

Thanks for pointing me to that story 👍 About your concern, I tried to pull your repo and it works on my end, seems there are some missing packages that I might not added to the CI. image Checking on that shortly

Nihisil commented 9 months ago

maybe for local trivy installation we need to do some setup? I just did asdf install and after that tried to use it through trivy config .

nvminhtue commented 9 months ago

maybe for local trivy installation we need to do some setup? I just did asdf install and after that tried to use it through trivy config .

Yes, that's pretty much enough, it should work after then 🤔

Nihisil commented 9 months ago

Yes, that's pretty much enough, it should work after then 🤔

I've reinstalled Trivy, and it's working smoothly on localhost now. Thanks for your patience 🙏

The only one remained concern for this PR is not working Trivy on CI for generated project

Nihisil commented 8 months ago

@nvminhtue please rebase this PR with develop branch to solve merge conflicts :pray: