search

canonical / ubuntu-com-security-api

The API for CVEs and USNs data.
17 stars 9 forks source link

notices.json "details" parameter not returning matches known related CVEs. #36

Closed blackboxsw closed 3 years ago

blackboxsw commented 3 years ago

Summary

Hidden USNs are being shown by default on public CVE queries. Not certain if USNs should remain hidden from CVE queries by default unless an additional parameter is provided. Also when trying to search from notices.json API providing the same CVE in the "details" parameter, no matches are found with or without hidden params.

Process

curl -X GET "https://staging.ubuntu.com/security/cves/CVE-2017-9233.json" -H "accept: application/json"
...
"notices": [                                                                    
   "USN-6100-1"                                                                 
]  

# No USNs returned from notices.json API route when searching for existing CVE with and without the hidden param
curl -X GET "https://staging.ubuntu.com/security/notices.json?details=CVE-2017-9233" -H "accept: application/json"
{"limit":20,"notices":[],"offset":0,"total_results":0}

# In production, related CVEs can be searched from notices.json API route using "details" param:
curl -X GET "https://ubuntu.com/security/notices.json?details=CVE-2017-9233" -H "accept: application/json"
{ USN-3356-1 ... USN-3356-2... "total_results":2}

Current and expected result

Current results were empty when querying security/notices.json for a known CVE related to a hidden USN in either the public query or the extra hidden param query.

Results from notices.json?details=CVE-id should contain any published USN related to CVE-id without any hidden USNs. When the extra hidden param is provided to the above query, both published and hidden USNs should be returned.

SirSamTumless commented 3 years ago

@albertkol Please could you review and triage this?

albertkol commented 3 years ago

@blackboxsw

I looked at the two issues you raised. 1) Hidden USNs show up on public CVE queries. How would you want this fixed? Would you want CVE read endpoints have a flag show_hidden too? Or would it be enough to always hide them?

2) No USNs returned from notices.json API route when searching for existing CVE with and without the hidden param I do not think this is an issue. You searched for notices that are linked to the cve_id from details and got 0 results. You got no results because the only USN that is linked to that CVE is a hidden one. Once the show_flag was added you got one result.

I think what confused you is that staging and prod do not have the same data. Prod returned you USN results even without the flag because prod has two public USNs linked to CVE CVE-2017-9233.

blackboxsw commented 3 years ago

Thanks @albertkol .

For 1. I think a feature request the allow for a hidden param set to true on the /security/cves.json and /security/cves/{cve_id}.json which would also append related hidden USNs to the list of published "notices". Without "hidden" true, only publlished USNs would be listed.

For 2. Apologies for the unclear issue report I did here. I get no results from staging for either hidden or not hidden queries against notices.json?details=CVE-2017-9233 https://pastebin.canonical.com/p/rZSm3386gK/

albertkol commented 3 years ago

@blackboxsw hmm.. I will have to investigate 2. I was so sure I got results from adding the flag earlier. My mistake.

How urgent are these two? Just to know how to prioritise things on our end.

blackboxsw commented 3 years ago

No worries, I do get valid responses in production for published CVEs/USNs via notices.json/detauls=CVE-id. So, maybe that's the issue. We're seeing data in some places, but not staging with hidden USNs for some reason.

Feature 1 may be unnecessary if Feature 2 works as expected. If ua client can use the notices.json route to query for hidden and/or published related CVEs, then we may not need "feature 1" because we can determine whether to honor a USN from either the notices.json route or a getting 404 on a response for a specific hidden USN or notices.json not returning hidden USN if we are missing a hidden param.

As far as priority, we ultimately can't really start testing ua fix on pre-release bits until we have one or the other features. We can test and validate published CVEs/USNs to date so we aren't blocked. But it'd inform our architecture to know if we can rely on notices.json?details=CVE-2017-9233

I'll have to defer on prioritization for a couple days to meet w/ security and the team and discuss our goals for pre-release USN testing.

albertkol commented 3 years ago

@blackboxsw

So you are saying Feature 1 is not necessary if Feature 2 works.

I will take a look at Feature 2. I hope it's a quick fix. If fixing Feature 2 makes Feature 1 unnecessary, then we can tackle Feature 1 only when you guys think it's mandatory.

Feature 1 related.. right now we always return all USN of a CVE, hidden or not. Would returning only public USN be better?

albertkol commented 3 years ago

@blackboxsw

I fixed the Feature 2 bug. Second query form your pastebin works now: https://pastebin.canonical.com/p/rZSm3386gK/

I left Feature 1 unchanged for now.