ossf / scorecard-action

Official GitHub Action for OpenSSF Scorecard.
Apache License 2.0
255 stars 70 forks source link

use Scorecard library entrypoint instead of Cobra hooking #1423

Closed spencerschrock closed 1 month ago

spencerschrock commented 1 month ago

Scorecard V5 released a new entrypoint, so make use of it instead of hooking into the underlying Cobra CLI. This gives us more flexibility when running Scorecard, such as writing the result to multiple formats.

The code got moved around a lot, my hope is it's organized more clearly now, but I'll try to summarize it for the sake of the reviewer.

entrypoint/entrypoint.go used to do several things

  1. Read the Scorecard Action options, which now takes place in our main.go https://github.com/ossf/scorecard-action/blob/486e8de2dca09b3f1cfac2bbec765156de6795a1/main.go#L35-L39

  2. All sorts of pre/post hooks to redirect output to both stdout and a result file. Which is done explicitly in a new format helper function. https://github.com/ossf/scorecard-action/blob/486e8de2dca09b3f1cfac2bbec765156de6795a1/internal/scorecard/format.go#L47-L53

signing/signing.go re-ran scorecard via GetJSONScorecardResults, which is now a simple extra formatting call.

https://github.com/ossf/scorecard-action/blob/486e8de2dca09b3f1cfac2bbec765156de6795a1/main.go#L54-L58

In terms of testing, separating the format code makes it more testable! But the rest of it is very hard to unit test. I'll look into testing the rest of it.

Fixes #1071

spencerschrock commented 1 month ago

In terms of testing, separating the format code makes it more testable! But the rest of it is very hard to unit test. I'll look into testing the rest of it.

Testing the Dockerfile locally, the JSON output is identical (using scdiff to compare), and the SARIF matches. The nightly e2e tests would test for fatal errors after merge as well.

But there aren't any great e2e tests currently setup before merge.