search

aboutcode-org / scancode.io

ScanCode.io is a server to script and automate software composition analysis pipelines with ScanPipe pipelines. This project is sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase/ Google Summer of Code, nexB and others generous sponsors!
https://scancodeio.readthedocs.io
Apache License 2.0
119 stars 88 forks source link

Allow rerunning pipelines #889

Open Hritik14 opened 1 year ago

Hritik14 commented 1 year ago

Currently, when a project is created and pipelines added, the user can run those pipelines only once. After one run is finished and the data gathered, there's no way to re-run the pipeline. If SCIO is put in a CI/CD, it'd need to run on every code build. It can currently be achieved by creating a project on the first CI/CD run and then resetting the project on every consequent run but this discards data from all the previous runs and does not maintain any history.

Hritik14 commented 1 year ago

Further, if a pipeline is reset and rerun from a CI in parallel many times (image multiple pushes causing multiple builds) it could bring in race conditions.

tdruez commented 1 year ago

You want to create a new project since your input changes between each CI run.

pombredanne commented 1 year ago

@Hritik14 I second this.... a project here is a ScanCode.io analysis project. Not necessarily an upstream project. Furthermore this is a snapshot in time.

Hritik14 commented 1 year ago

Makes sense... Although I feel there's a need to document this along with other measures required for a functional CI integration (eg: https://github.com/nexB/scancode.io/issues/863)

Let's add the docs label for now and we can come back to it later

tdruez commented 1 year ago

@Hritik14 the Automation/CI is quite new and a bit light at the moment: https://scancodeio.readthedocs.io/en/latest/automation.html

PR welcome ;)

Hritik14 commented 1 year ago

@pombredanne @tdruez

a project here is a ScanCode.io analysis project. Not necessarily an upstream project

Does scancodeio not keep the relationship between an upstream project -> scancodeio analysis project ? This would help in generating some interesting data like which of the upstream projects being scanned have the least vulnerability, maximum license violations etc. It helps in generating some sort of Score for the repository in the long run (looking at you, ossf scorecard).

tdruez commented 1 year ago

Does scancodeio not keep the relationship between an upstream project -> scancodeio analysis project ?

No, there's no concept of upstream project at the moment in ScanCode.io