Fix some issues with the import, specifially the SPDX SBOM import

ctron commented 1 week ago

The catalyst of this was https://github.com/trustification/trustify/issues/498 … working on this uncovered a few extra issues.

The main motivation was to fix the issue that an SBOM with broken references would (rightfully) cause a DB foreign key error. This issue can be be detected and a proper error message can be provided.

However, it also turned out that such an error would prevent the import from continuing and would simply abort. This is also fixed now, and such errors will be recorded as part of the report and stored in the DB at the end up the run.

Furthermore, the jobs used the "last run" time for the next run (as "since"). Which leads to the case that the previous documents, skipped due to the error, would be skipped again. This is changed in way that only a successful run can provide a timestamp for the next run. (Critically) failed runs will cause a complete re-run the next time the job is run.

Closes #498

ctron commented 1 week ago

@carlosthe19916 this is the PR with requires https://github.com/trustification/trustify-ui/pull/101

ctron commented 1 week ago

Blocked by https://github.com/KenDJohnson/cpe-rs/pull/15

ctron commented 4 days ago

Blocked by KenDJohnson/cpe-rs#15

Created a PR and set that as a patch

ctron commented 4 days ago

rebased

trustification / trustify

Fix some issues with the import, specifially the SPDX SBOM import #511