search

aboutcode-org / scancode.io

ScanCode.io is a server to script and automate software composition analysis pipelines with ScanPipe pipelines. This project is sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase/ Google Summer of Code, nexB and others generous sponsors!
https://scancodeio.readthedocs.io
Apache License 2.0
115 stars 85 forks source link

Optionally fail CLI scan (return error code) on policy violation "error" #1346

Closed pombredanne closed 6 days ago

pombredanne commented 3 months ago

I would like to optionally fail a CLI scan by returning a failure error code on policy violation of type "error".

pombredanne commented 3 months ago

@silverhook ping

silverhook commented 3 months ago

cc @brittneyq

tdruez commented 2 months ago

Some design ideas following discussion with @pombredanne

In general, compliance "failure" should be distinguished from a pipeline execution failure. Encountering a compliance issue should not stop or make the pipeline fail, but rather be logged in the database as it is already done at the moment.

I would suggest a post-pipeline approach, adding a new check management command. By default, that command would make queries in the database looking for compliance issues for a given project, and "fail" returning a non-zero exit code. The command could also return of summary of the compliance issues. We can offer options to configure the check, such as the failure level, the output, the verbosity, and so on...

For example: --fail-level=WARNING (default: ERROR). The check command can be easily added to a CI/CD after the pipeline execution.

@pombredanne suggested that in place of a project, a JSON results file could be also provided as the input to "check". This be useful for compatibility with the current "run" implementation, where the data does not persist in the DB but is returned directly as JSON.

Also, some ideas to improve the visibility of the compliance issues in the UI/API:

tdruez commented 2 months ago

A new check-compliance management command is now available.

Documented at https://scancodeio.readthedocs.io/en/latest/command-line-interface.html#scanpipe-check-compliance-project-project

scanpipe check-compliance --project=myproject --fail-level=WARNING

476 compliance issues detected on this project.
Package:
 - warning: 455
 - error: 21
brittneyq commented 2 weeks ago

hi @tdruez , is this also available through API request or is that something that needs to be added?

cc: @pombredanne

tdruez commented 1 week ago

A new compliance API action was added in https://github.com/aboutcode-org/scancode.io/pull/1416 Documentation available at https://scancodeio.readthedocs.io/en/latest/rest-api.html#compliance

tdruez commented 6 days ago

A new Compliance panel in the project details view was added in #1417 Documentation available at https://scancodeio.readthedocs.io/en/latest/tutorial_license_policies.html#web-ui

The compliance alerts are now available in the CLI, UI, and REST API. I'm closing this one as done. Let's create new issues for further improvements.