search

aquasecurity / trivy-db

Apache License 2.0
215 stars 131 forks source link

feat: Expose Ubuntu fix status for downstream consumption #407

Closed skahn007gl closed 1 week ago

skahn007gl commented 3 months ago

Summary

resolves #408

Trivy Ubuntu advisories provide a FixedVersion when there is a released fix for a package, Affected can be inferred when the advisory is present without a FixedVersion, it does not expose any other status that Canonical use to indicate the status of a fix. This is insufficient to infer a status of ignored, pending or needed as these status show the package is affected and in the process of getting to a fixed version or not if the status is ignored.

This change exposes the Status provided in launchpad advisories without changed existing behaviour to populate the notes field in an advisory with Status:$status

Changes

CLAassistant commented 3 months ago

CLA assistant check
All committers have signed the CLA.

skahn007gl commented 3 months ago

Whilst I'm not a maintainer here, I am aware of the scope of this work and did work on the RHEL changes for fix status, so thought I'd add some review comments to try and help to save some time for the reviewers.

@jhebden-gl Thanks or the feedback. I have pushed a new change set with your suggested changes

jhebden-gl commented 2 months ago

Hi @knqyf263 👋🏻 - I was wondering if you might be able to assist with a review on this PR? We are hoping to use this data downstream. Thank you!

skahn007gl commented 2 months ago

fix linter error, please https://github.com/aquasecurity/trivy-db/actions/runs/9608945100/job/26502649422?pr=407#step:4:26

Done :)

knqyf263 commented 2 months ago

@skahn007gl Thanks for updating. At last, could you add a test with a non-fixed status? Adding a new advisory or updating the existing one.

knqyf263 commented 2 months ago

@skahn007gl Also, could you open a PR with this change in Trivy? It's better to test this change with Trivy before merging this PR.

Tips: Use replace until this PR gets merged.

skahn007gl commented 2 months ago

@skahn007gl Also, could you open a PR with this change in Trivy? It's better to test this change with Trivy before merging this PR.

Tips: Use replace until this PR gets merged.

Hey @knqyf263 I am unsure what needs to change in the existing integration tests. Is my understanding of the tests correct that it tests 2 scenarios?

  1. Include all findings
  2. Include findings with fixes only

This includes the status field in the .golden test data.

I have made the other changes and raised a PR to Trivy :) Trivy PR7020

knqyf263 commented 2 months ago

There is a DB fixture here. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/ubuntu.yaml

We need to add a status here and ensure the status appears in the result.

This bucket must also be updated if you want to add a new vulnerability. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/vulnerability.yaml

When adding a new bucket for Ubuntu, that needs to be added here. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/data-source.yaml#L137-L141

skahn007gl commented 2 months ago

There is a DB fixture here. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/ubuntu.yaml

We need to add a status here and ensure the status appears in the result.

This bucket must also be updated if you want to add a new vulnerability. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/vulnerability.yaml

When adding a new bucket for Ubuntu, that needs to be added here. https://github.com/aquasecurity/trivy/blob/8d618e48a2f1b60c2e4c49cdd9deb8eb45c972b0/integration/testdata/fixtures/db/data-source.yaml#L137-L141

@knqyf263 How are the .golden files generated/created?

I have updated the DB fixture with an existing CVE for ubuntu 18.04 that would have a non fixed status, but i'm unable to trigger a failure on the existing ubuntu18.04 integration tests.

- bucket: ubuntu 18.04
  pairs:
    - bucket: libspring-java
      pairs:
        - key: CVE-2022-22965
          value:
            Status: deferred
knqyf263 commented 2 months ago

deferred is not our status. https://github.com/aquasecurity/trivy-db/blob/9bdfe07de664866c0d889b2d1f8d6ecc11b640ad/pkg/types/status.go#L17-L26

skahn007gl commented 2 months ago

deferred is not our status.

https://github.com/aquasecurity/trivy-db/blob/9bdfe07de664866c0d889b2d1f8d6ecc11b640ad/pkg/types/status.go#L17-L26

Corrected to fix_deferred, but still no failure in the test run.

I am assuming I need to update ubuntu-1804.json.golden to update the expected results for the tests linked https://github.com/aquasecurity/trivy/blob/bbaf5952bc8059c537401bd4364e019eecd16c9a/integration/standalone_tar_test.go#L217-L233

If this is the case, How are the .golden files generated/created?

Thanks for the help and guidance on this as I work through it.

knqyf263 commented 2 months ago

I am assuming I need to update ubuntu-1804.json.golden to update the expected results for the tests linked https://github.com/aquasecurity/trivy/blob/bbaf5952bc8059c537401bd4364e019eecd16c9a/integration/standalone_tar_test.go#L217-L233

No, the test should fail due to a mismatch with the golden file. You need to update the golden file once you confirm the test fails. Did you update all the buckets I listed above? It may be easier for us to debug if you push the change to your PR in Trivy.

skahn007gl commented 2 months ago

I am assuming I need to update ubuntu-1804.json.golden to update the expected results for the tests linked https://github.com/aquasecurity/trivy/blob/bbaf5952bc8059c537401bd4364e019eecd16c9a/integration/standalone_tar_test.go#L217-L233

No, the test should fail due to a mismatch with the golden file. You need to update the golden file once you confirm the test fails. Did you update all the buckets I listed above? It may be easier for us to debug if you push the change to your PR in Trivy.

Thanks for confirming, I agree it will be easier to debug if i share my changes. I will update the PR shortly.

skahn007gl commented 2 months ago

@knqyf263 I have pushed the updated DB fixture and OS bucket. I used an existing vulnerability.

Note the changes do not include the use of replace in go.mod, but i do use it locally. e.g: replace github.com/aquasecurity/trivy-db => /xxx/yyy/zzz/trivy-db

knqyf263 commented 2 months ago

You added Ubuntu 21.10, but the scanned image is not Ubuntu 21.10. The data is not used.

skahn007gl commented 2 months ago

You added Ubuntu 21.10, but the scanned image is not Ubuntu 21.10. The data is not used.

Adjusted to ubuntu 16.04 as there is an image under integration/testdata/fixtures/images Added test for ubuntu 16.04 into integration/standalone_tar_test.go note: Test does not use the correct json.golden as there is no existing one for ubuntu 16.04

Successfully generated a failed test result.

I am running the tests via mage test:integration(mage is new to me) or go test -timeout 15m -v -tags=integration ./integration/... But i do not see any test artefacts generated, only pass/fail stat's.

Is there a way to generate the output from the comparison?.

I have pushed my updates.

skahn007gl commented 2 months ago

Hey @knqyf263 Sorry for the delays on my side, I've had some personal life stuff crop up.

I have simplified the integration test to use as much as existing as possible to understand how these tests hang together, I believe I am still missing something here.

When i run the integration test's, there is no test failure for ubuntu-1804

I added some dummy data to to ubuntu-1804.json.golden This induces an error, but the error indicates it did not expect the new vulnerability.

    -   Vulnerabilities: ([]types.DetectedVulnerability) (len=6) {
    +   Vulnerabilities: ([]types.DetectedVulnerability) (len=5) {