CSAF Importer does not work

[carlosthe19916]

Steps to reproduce:

• Disable auth (just to allow our terminal to reach the endpoints easily)

  export AUTH_DISABLED=true

• Start the server (make sure you clean your .trustify directory to be sure you have a clean instance running)

  cargo run --bin trustd

• Create a CSAF importer:

http POST localhost:8080/api/v1/importer/redhat-csaf csaf[source]=https://redhat.com/.well-known/csaf/provider-metadata.json csaf[disabled]:=false csaf[period]=30s csaf[v3Signatures]:=true

Look at the server log and you will see something like:

2024-06-06T11:09:45.470058Z  INFO run:run_once:run_once_csaf: trustify_module_importer::server::csaf: return=Err(Normal { err: Visitor error: Query Error: error returned from database: insert or update on table "cvss3" violates foreign key constraint "cvss3_vulnerability_id_fkey", output: RunOutput { report: Report { start_date: 2024-06-06 11:09:43.960618328 +00:00:00, end_date: 2024-06-06 11:09:45.470015842 +00:00:00, numer_of_items: 1, messages: {Upload: {"https://access.redhat.com/security/data/csaf/v2/advisories/2024/rhsa-2024_3666.json": [Message { severity: Error, message: "upload failed: Query Error: error returned from database: insert or update on table \"cvss3\" violates foreign key constraint \"cvss3_vulnerability_id_fkey\"" }]}} }, continuation: None } }) importer=CsafImporter { common: CommonImporter { disabled: false, period: 30s, description: None }, source: "https://redhat.com/.well-known/csaf/provider-metadata.json", v3_signatures: true, only_patterns: [] } last_run=None
2024-06-06T11:09:45.470281Z ERROR run:run_once: trustify_module_importer::server: error=Visitor error: Query Error: error returned from database: insert or update on table "cvss3" violates foreign key constraint "cvss3_vulnerability_id_fkey"
2024-06-06T11:09:45.470360Z  INFO run: trustify_module_importer::server: Import run complete: Some("Visitor error: Query Error: error returned from database: insert or update on table \"cvss3\" violates foreign key constraint \"cvss3_vulnerability_id_fkey\"")    

Another thing

• The README file indicates that the source of a CSAF file for red hat should be https://redhat.com/.well-known/csaf/provider-metadata.json
• On the other hand, the sample data indicates that the source of CSAF is redhat.com

https://github.com/trustification/trustify/blob/bc9e6a5b226b8e11a4b44d1315ae003546454a0b/server/src/sample_data.rs#L97

One of them is not correct, either the README or the sample data. But in both cases the original error described in this issue applies to both

[ctron]

Both variants (redhat.com as well as the full URL) work.

The error looks more like a general ingest error. Which is just the service used by the importer. So I would recommend testing this with a plain upload.

[carlosthe19916]

I have made a quick test and upload a CSAF file manually and it worked. So the issue might be the importer.

The title of this issue refers to the importer not working for CSAF files so the issue is still valid. And the steps to reproduce it are also valid

[ctron]

You can easily reproduce the issue like this:

➜  repro wget https://access.redhat.com/security/data/csaf/v2/advisories/2024/rhsa-2024_3666.json
rhsa-2024_3666.json  100% [====================================================================================================================================================================>]    4.50K    --.-KB/s
                          [Files: 1  Bytes: 4.50K [21.26KB/s] Redirects: 0  Todo: 0  Errors: 0                                                                                                  ]
➜  repro cat rhsa-2024_3666.json | http POST localhost:8080/api/v1/advisory Authorization:$(oidc token trusty -b)
HTTP/1.1 400 Bad Request
access-control-allow-credentials: true
access-control-expose-headers: content-type
content-encoding: br
content-type: application/json
date: Thu, 06 Jun 2024 11:52:54 GMT
transfer-encoding: chunked
vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
vary: accept-encoding

{
    "error": "Ingestor error",
    "message": "Query Error: error returned from database: insert or update on table \"cvss3\" violates foreign key constraint \"cvss3_vulnerability_id_fkey\""
}

It has nothing to do with the importer.

[jcrossley3]

Confirmed the bug locally. Any advisory having more than one vulnerability seems to trigger it. Thanks for the test case!