Our tests often fail because APIs are unavailable because of maintenance, changes in its call signature or just tired from dealing with our requests. This is annoying, so we would ideally change our testing to split the reading of data from the testing of the API call.
Basic idea:
separate the API calls from the reading logic (perhaps also useful for allowing multi-threaded downloading as was implemented for Lizard?).
test all reading logic on pre-downloaded files (e.g. BRO XML, KNMI precipitation txt, etc.)
minimize the API tests to only test the response from the server and whether the returned data can be parsed
mark the API tests to allow-fail so we do get a warning when these tests fail but it isnt necessarily a block on development
Our tests often fail because APIs are unavailable because of maintenance, changes in its call signature or just tired from dealing with our requests. This is annoying, so we would ideally change our testing to split the reading of data from the testing of the API call.
Basic idea: