Adjust CLI logging level

[rgreinho]

By default the CLI shows a bunch of detailed errors:

./chain-bench scan --repository-url https://github.com/buildsec/frsca --access-token $GITHUB_TOKEN -o ./results
2022-06-17 09:10:07 INF 🚩 Fetch Starting
2022-06-17 09:10:08 ERR error in fetching organization hooks error="GET https://api.github.com/orgs/buildsec/hooks: 404 Not Found []"
2022-06-17 09:10:08 INF 🏢 Fetching Organization Settings Finished
2022-06-17 09:10:08 ERR error in fetching org packages error="GET https://api.github.com/orgs/buildsec/packages?package_type=npm&state=active: 403 You need at least read:packages scope to list packages. []"
2022-06-17 09:10:13 ERR error in fetching hooks data error="GET https://api.github.com/repos/buildsec/frsca/hooks: 404 Not Found []"
2022-06-17 09:10:13 INF 🛢️ Fetching Repository Settings Finished
2022-06-17 09:10:13 ERR error in fetching branch protection error="GET https://api.github.com/repos/buildsec/frsca/branches/main/protection: 404 Not Found []"
2022-06-17 09:10:13 INF 🌱 Fetching Branch Protection Settings Finished
2022-06-17 09:10:13 INF 👫 Fetching Members Finished
2022-06-17 09:10:13 WRN file .github/workflows/ci.yaml not found
2022-06-17 09:10:14 WRN file dynamic/pages/pages-build-deployment not found
2022-06-17 09:10:14 INF 🔧 Fetching Pipelines Finished
2022-06-17 09:10:14 INF 🏁 Fetch succeeded

But these errors just clutter the output, and are not very useful unless debugging information is needed. These types of details should be displayed for debug or tracing log level.

As a user I should be able to adjust the log level from the CLI, for example by supplying -v flags (1 for info, 2 for debug, 3 for trace).

[josearomeroj]

Something like this?

https://github.com/aquasecurity/chain-bench/blob/45436aeac3894dd20218f89c06e9fc8d16c21a1b/internal/commands/flags.go#L20-L22

[naortalmor1]

@rgreinho Thanks again for your contributions! So just to clarify, your suggestion is to remove the full error message from the printer when we get an error?

[rgreinho]

Oh no, not at all. I was actually vouching for a more gradual approach to display the errors, possibly adding some context to resolve the issue.

For instance, the line:

2022-06-17 09:10:08 ERR error in fetching organization hooks error="GET https://api.github.com/orgs/buildsec/hooks: 404 Not Found []"

Could become:

2022-06-17 09:10:08 ERR error in fetching organization hooks"
2022-06-17 09:10:08 ERR check token permission: require the admin:org_hook scope (https://docs.github.com/en/rest/orgs/webhooks#scopes-and-restrictions=)"
2022-06-17 09:10:08 DEBUG organization hooks error="GET https://api.github.com/orgs/buildsec/hooks: 404 Not Found []"

Where the ERR level would provide the error description, a potential resolution, and the DEBUG level would provide more useful details for debugging if needed.

Regarding my remark about the CLI, I was advocating in favor of changing the behavior of the -v flag to use it to define the verbosity level. I don't have a Golang example handy, but the concept can be seen in this Rust project:

• the CLI parser counts the number of -v provided on the command line (https://github.com/rgreinho/labelr-rs/blob/master/src/cli.rs#L8-L10=)
• the main entrypoint sets the logging level accordingly (https://github.com/rgreinho/labelr-rs/blob/master/src/main.rs#L20-L27=)

I find this approach more ergonomic than using --log-level debug, but I do realize that this may be a personal preference, and since the functionality is already there, it may not be that important actually.

[LiorVais]

Thanks Remy! @rgreinho I like the idea of using the verbosity flag to determine the log level, it doesn't have much use on its own anyway. I'm also thinking to use the quiet flag as the opposite of verbose, to adjust the log level the other way. Fix coming soon, with log adjustments as well.