Testing different producers

[sustefil]

Each of the producer mostly has the same format or/and errors in validation, thus I pick just some examples

This was a manual testing with commit https://github.com/csaf-tools/CVRF-CSAF-Converter/commit/867578e5f8263cc908aa675d0fa75ea9a988b5ba. Just to see if we have some conversion errors. Could be automated in the future.

Used producer files: https://github.com/csaf-tools/CVRF-CSAF-Converter/tree/testing/examples

And this helper script: https://github.com/csaf-tools/CVRF-CSAF-Converter/blob/testing/tests/test_producers.sh

RedHat - ALL invalid input

Invalid cvrf2doc namespace

INPUT FILE: examples/examples_redhat/RHSA-2021:2040.xml 
2022-02-18 15:04:29,499 - utils - CRITICAL - Input document not valid: Element '{http://www.icasi.org/CVRF/schema/cvrf/1.1}cvrfdoc': No matching global declaration available for the validation root. (<string>, line 0).

Fortiguard - ALL invalid input

Missing mandatory fields

INPUT FILE: examples/examples_fortiguard/FG-IR-21-192.xml 
2022-02-18 15:06:34,935 - utils - CRITICAL - Input document not valid: Element '{http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/cvrf}CurrentReleaseDate': This element is not expected. Expected is ( {http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/cvrf}Status ). (<string>, line 0).

Suse

28 examples OK

INPUT FILE: examples/examples_suse/1.2/cvrf-opensuse-su-2015%3A1968-1.xml 
2022-02-18 15:46:25,227 - cvrf2csaf - INFO - CSAF schema validation OK

210 examples with ERRORs (not conversion errors, but input errors)

2022-02-18 15:30:20,600 - vulnerability - ERROR - No product_id entry for CVSS score set.
2022-02-18 15:30:20,856 - cvrf2csaf - ERROR - CSAF schema validation error. Path: $.vulnerabilities[0].scores[0].products. Message: [] is too short.

282 examples with CRITICAL e.g. missing ProductID in Vulnerabilities

2022-02-18 15:49:27,111 - utils - CRITICAL - Input document not valid: Element '{http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/vuln}Status': Missing child element(s). Expected is ( {http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/vuln}ProductID ). (<string>, line 0).

Cisco

100 examples OK - took first 100 files from the batch and all of them were valid, no errors.

Siemens

5 examples OK - Haven't found any "browsing" page for Siemens CVRFs, managed to google a few examples, all of them valid, no errors.

Oracle

3 examples with CRITICAL - wrong cvrfdoc namespace used

2022-02-28 17:13:30,646 - cvrf2csaf - ERROR - Errors during input validation occurred, reason(s): [git/CVRF-CSAF-Converter/examples/examples_oracle/cpujan2022cvrf.xml:4:0:ERROR:SCHEMASV:SCHEMAV_CVC_ELT_1: Element '{http://www.icasi.org/CVRF/schema/cvrf/1.1}cvrfdoc': No matching global declaration available for the validation root.].
2022-02-28 17:13:30,646 - utils - CRITICAL - Input document not valid, reason(s).

[sustefil]

Hello @tschmidtb51 , please have a look.

Long story short, the input validation is pretty strict and most of the producers CVRFs don't adhere to it.

From the conversion of valid inputs, we haven't observed any conversion errors so far.

[tschmidtb51]

Please also check Siemens and Oracle.

Long story short, the input validation is pretty strict and most of the producers CVRFs don't adhere to it.

I'll have a look at that.

From the conversion of valid inputs, we haven't observed any conversion errors so far.

Sounds great.

[sustefil]

All producers above tested, can be closed from my side.