gocsaf / csaf

Tools to download or provide CSAF (Common Security Advisory Framework) documents.
https://csaf.io
40 stars 23 forks source link

Unable to download Microsoft CSAF #588

Open Rafiot opened 2 days ago

Rafiot commented 2 days ago

I'm not sure what the issue is, but fetching the CSAF entries from Microsoft fails in a weird way: downloader.log

{"time":"2024-11-19T15:19:47+01:00","level":"INFO","msg":"AdvisoryFileProcessor.Process: \"https://msrc.microsoft.com/csaf/changes.csv\" has an invalid time stamp in line 89: parsing time \"2024-11-12T08:00:00.0000000\" as \"2006-01-02T15:04:05Z07:00\": cannot parse \"\" as \"Z07:00\""}

The call:

csaf_downloader -d csaf_microsoft https://msrc.microsoft.com/csaf/provider-metadata.json

Do you have any idea how to fix it?

bernhardreiter commented 1 day ago

A line from https://msrc.microsoft.com/csaf/changes.csv:

2024/msrc_cve-2024-49060.json,2024-11-15T08:00:00.0000000

Checking the format, this seems to be missing a timezone, so it is an invalid file.

https://docs.oasis-open.org/csaf/csaf/v2.0/os/csaf-v2.0-os.html#7113-requirement-13-changescsv has

lines MUST be sorted by the current_release_date timestamp

https://docs.oasis-open.org/csaf/csaf/v2.0/os/csaf-v2.0-os.html#321122-document-property---tracking---current-release-date

(current_release_date) with value type string with format date-time

And date-time is defined in https://json-schema.org/understanding-json-schema/reference/string as

Dates and times are represented in RFC 3339, section 5.6.

"date-time": Date and time together, for example, 2018-11-13T20:20:39+00:00

https://datatracker.ietf.org/doc/html/rfc3339#section-5.6 then has an ABNF which has

time-offset = "Z" / time-numoffset full-time = partial-time time-offset

The time-offset is missing in the file. Best would be if the provider could fix this on their end.

Rafiot commented 13 hours ago

I'm reporting it to Microsoft, and pointing them to your details, thanks!

Rafiot commented 4 hours ago

Update on that: the time issue is fixed, but now I get this error:

{"time":"2024-11-21T22:39:59+01:00","level":"WARN","msg":"Content type is not 'application/json'","url":"https://msrc.microsoft.com/csaf/2024/msrc_cve-2024-49028.json","content_type":"application/json; charset=utf-8"}

Their server returns application/json; charset=utf-8 when you expect application/json, but both should be valid (?).

I couldn't find it in the specs, but I'll drop them another mail if needed.

tschmidtb51 commented 2 hours ago

Their server returns application/json; charset=utf-8 when you expect application/json, but both should be valid (?).

True. IMHO, both should be valid. Flagging @bernhardreiter to suggest on how we want to deal with the situation in the checker in general.