enterprise-contract / ec-cli

Enterprise Contract command line interface
https://enterprisecontract.dev/docs/ec-cli/main/index.html
Apache License 2.0
27 stars 29 forks source link

Rework validate image human readable text output #1656

Closed simonbaird closed 4 months ago

simonbaird commented 4 months ago

Ref: https://issues.redhat.com/browse/EC-657

simonbaird commented 4 months ago

Some example output:

With one component (and without --info):

Success: false                                                                                                                                                                                                                                  
Result: FAILURE                                                                                                                                                                                                                                 
Violations: 1, Warnings: 9, Successes: 82                                                                                                                                                                                                       
Component: golden-container                            
ImageRef: registry.redhat.io/rhtas/ec-rhel9@sha256:fc5c65c2829f170ecc5c0ed33b154c9afde3b3b1f347766a66c4af65d343ea8c

Results:
✕ [Violation] hermetic_build_task.build_task_hermetic
  ImageRef: registry.redhat.io/rhtas/ec-rhel9@sha256:fc5c65c2829f170ecc5c0ed33b154c9afde3b3b1f347766a66c4af65d343ea8c
  Reason: Build task was not invoked with the hermetic parameter set

› [Warning] cve.deprecated_cve_result_name
  ImageRef: registry.redhat.io/rhtas/ec-rhel9@sha256:fc5c65c2829f170ecc5c0ed33b154c9afde3b3b1f347766a66c4af65d343ea8c
  Reason: CVE scan uses deprecated result name

  ...
codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 0% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 86.68%. Comparing base (4e3a778) to head (3641f75).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/graphs/tree.svg?width=650&height=150&src=pr&token=CY5ORXJB33&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract)](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) ```diff @@ Coverage Diff @@ ## main #1656 +/- ## ========================================== + Coverage 80.13% 86.68% +6.54% ========================================== Files 65 76 +11 Lines 4788 5136 +348 ========================================== + Hits 3837 4452 +615 + Misses 951 684 -267 ``` | [Flag](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | Coverage Δ | | |---|---|---| | [acceptance](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | `71.94% <0.00%> (?)` | | | [generative](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | `80.00% <0.00%> (-0.14%)` | :arrow_down: | | [integration](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | `80.00% <0.00%> (-0.14%)` | :arrow_down: | | [unit](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | `80.00% <0.00%> (-0.14%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract) | Coverage Δ | | |---|---|---| | [internal/utils/templates.go](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656?src=pr&el=tree&filepath=internal%2Futils%2Ftemplates.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract#diff-aW50ZXJuYWwvdXRpbHMvdGVtcGxhdGVzLmdv) | `55.00% <0.00%> (-1.90%)` | :arrow_down: | | [internal/applicationsnapshot/report.go](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656?src=pr&el=tree&filepath=internal%2Fapplicationsnapshot%2Freport.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract#diff-aW50ZXJuYWwvYXBwbGljYXRpb25zbmFwc2hvdC9yZXBvcnQuZ28=) | `78.08% <0.00%> (+7.74%)` | :arrow_up: | ... and [40 files with indirect coverage changes](https://app.codecov.io/gh/enterprise-contract/ec-cli/pull/1656/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=enterprise-contract)
simonbaird commented 4 months ago

With two components and --info on:

Success: false                         
Result: FAILURE                                                                                                                                                                                                                                 
Violations: 6, Warnings: 18, Successes: 158

Components:
- Name: golden-container-sha256:b4b23d19b57ea0333f89fb1fa781df57c3cb00f284bd7a384fac093cacf31e58-amd64
  ImageRef: quay.io/redhat-appstudio/ec-golden-image@sha256:b4b23d19b57ea0333f89fb1fa781df57c3cb00f284bd7a384fac093cacf31e58
  Violations: 3, Warnings: 9, Successes: 79

- Name: golden-container-sha256:79afd494c4d1f3c3e1ba328a20df36e79377db06da94152dd2456f0259e42b98-arm64
  ImageRef: quay.io/redhat-appstudio/ec-golden-image@sha256:79afd494c4d1f3c3e1ba328a20df36e79377db06da94152dd2456f0259e42b98
  Violations: 3, Warnings: 9, Successes: 79

Results:
✕ [Violation] redhat_manifests.redhat_manifests_missing
  ImageRef: quay.io/redhat-appstudio/ec-golden-image@sha256:b4b23d19b57ea0333f89fb1fa781df57c3cb00f284bd7a384fac093cacf31e58
  Reason: Missing Red Hat manifest "root/buildinfo/content_manifests/sbom-purl.json"
  Title: Missing Red Hat manifests
  Description: Verify the expected Red Hat manifests are available in the image. To exclude this rule add
  "redhat_manifests.redhat_manifests_missing:root/buildinfo/content_manifests/sbom-purl.json" to the `exclude` section of the
  policy configuration.

✕ [Violation] sbom_cyclonedx.found
  ImageRef: quay.io/redhat-appstudio/ec-golden-image@sha256:b4b23d19b57ea0333f89fb1fa781df57c3cb00f284bd7a384fac093cacf31e58
  Reason: No CycloneDX SBOM found
  Title: Found
  Description: Confirm a CycloneDX SBOM exists. To exclude this rule add "sbom_cyclonedx.found" to the `exclude` section of the
  policy configuration.
  Solution: Make sure the build process produces a CycloneDX SBOM.

  ...