canonical / ubuntu-com-security-api

The API for CVEs and USNs data.
17 stars 9 forks source link

Made notice and cve queries case insensitive #74

Closed mtruj013 closed 2 years ago

mtruj013 commented 2 years ago

Done

QA

Issue / Card

Fixes https://github.com/canonical-web-and-design/ubuntu-com-security-api/issues/60

nottrobin commented 2 years ago

The CVE tests are now broken, which is odd... https://github.com/canonical-web-and-design/ubuntu-com-security-api/runs/5021148875?check_suite_focus=true

nottrobin commented 2 years ago

@mtruj013 so I can walk you through how to debug the tests yourself. I should probably do this tomorrow if I get a chance.

But the error you're getting is because the .get(cve_id) way of retrieving CVEs doesn't seem to be returning anything, according to the test at least. Does it definitely work in real life? In which case the testing method is kinda broken. Or is it maybe actually broken? In which case we need to revert the code to the old .filter() method.

mtruj013 commented 2 years ago

@nottrobin Ok, I tried everything I could locally as it was before and then with the changes and then did the same with the actual tests. Here are the results:

nottrobin commented 2 years ago

Ok so tomorrow let's look into the disparity between the python test and your actual testing for retrieving CVEs. That sounds like a false negative which isn't good. We shouldn't have testing infrastructure that can return a false negative like that.

nottrobin commented 2 years ago

It looks like the current testing uses sqlalchemy.testing.mock and magicmock to basically create a bunch of fake function responses based on specific calls that the tests make. This, if course, doesn't really test the calls themselves, it tests the mocks, which isn't all that useful.

I think a better approach might be to use e.g. testing.postgresql (https://eradman.com/posts/ephemeral-databases.html) to create an ephemeral database so we can be sure sqlalchemy is behaving as it would in the real world.

But let's look into this more deeply tomorrow.