This PR addresses the issue with our weekly updater receiving a 404 when trying to grab metadata. Two of the buckets we draw data from were updated and a file we used to rely on is no longer there. It is possible to get the same data from other means though.
I was hoping that this meant that all three buckets can get their metadata in the same way, and we could condense the functions down to one main set, but unfortunately that is not the case. One of our buckets is still just slightly different from the rest (namely, that it has a finished.json but this file contains less metadata than the other two's finished.json so you have to do some spelunking to find out which kubernetes version is being used). So I kept the style of each bucket having its own set of functions for metadata retrieval because, while unpleasantly verbose, it better illustrates that we cannot expect these three to behave the same. Any of them could change at any moment, it seems.
This PR addresses the issue with our weekly updater receiving a 404 when trying to grab metadata. Two of the buckets we draw data from were updated and a file we used to rely on is no longer there. It is possible to get the same data from other means though.
I was hoping that this meant that all three buckets can get their metadata in the same way, and we could condense the functions down to one main set, but unfortunately that is not the case. One of our buckets is still just slightly different from the rest (namely, that it has a
finished.json
but this file contains less metadata than the other two'sfinished.json
so you have to do some spelunking to find out which kubernetes version is being used). So I kept the style of each bucket having its own set of functions for metadata retrieval because, while unpleasantly verbose, it better illustrates that we cannot expect these three to behave the same. Any of them could change at any moment, it seems.