The code is in a try-except block which catches all HTTPErrors but only does anything with 404s. In general we should not silently ignore things like this. The code is written such that when an error occurs and is ignored, important steps like deduplication and "local" filtering never happen (since they were located inside the try-except block), so the query results wind up being mysteriously incorrect.
In particular, I found a problem with 416 (Range Not Satisifiable). That status code shouldn't necessarily be interpreted as an error at all. It can naturally occur during TAXII 2.0 paging when you run past the end of the pages. Running past the end of the pages can occur for example, whenever the server isn't sending Content-Range headers, so the only way for the client to know when it has hit the end is to run off the end and not get any results. Currently when this happens, the error is silently ignored, the deduplication and local-filtering steps are skipped, and you get incorrect results.
The code is in a try-except block which catches all HTTPErrors but only does anything with 404s. In general we should not silently ignore things like this. The code is written such that when an error occurs and is ignored, important steps like deduplication and "local" filtering never happen (since they were located inside the try-except block), so the query results wind up being mysteriously incorrect.
In particular, I found a problem with 416 (Range Not Satisifiable). That status code shouldn't necessarily be interpreted as an error at all. It can naturally occur during TAXII 2.0 paging when you run past the end of the pages. Running past the end of the pages can occur for example, whenever the server isn't sending
Content-Rangeheaders, so the only way for the client to know when it has hit the end is to run off the end and not get any results. Currently when this happens, the error is silently ignored, the deduplication and local-filtering steps are skipped, and you get incorrect results.