Explorers show no data when variable IDs or csv file URLs change after a dataset update

[pabloarosado]

Description

Indicator-based and csv-based explorers rely on variable IDs and csv file URLs. When they change, they may silently start showing no data and we have no way to know.

Expected behaviour

We should be warned when an indicator-based explorer is relying on indicators or csv files that do not exist anymore (or even better, also when they are not the last version).

Additional context

In an ideal world, we should rethink how the inputs of explorer tsv files should be defined, but, given that this year we are probably not going to create many explorers, for now, at least, we should be aware when an explorer fails to load data.

[ikesau]

There are a few places we could try to address this:

1. Add validation in mirror_explorers
2. Add validation during baking
3. Add better bugsnag reporting when an explorer tries to load a variable via an invalid ID
   • Potentially hard to reliably narrow down to this specific failure case when loading can fail due to wifi/firewalls/etc

[pabloarosado]

On the ETL side, one thing we can do (and is happening already for several explorers) is to use version latest (instead of a date) on explorer steps. This way, nothing needs to be changed on csv-based explorers. So, if the data changes in ETL, the URL to the csv file feeding the explorer will be the same, and therefore the data in the explorer will change automatically.

This avoids the problem of csv-based explorers showing no data, but it doesn't solve the problem for indicator-based explorers.

[sophiamersmann]

3552 added bugsnag reporting when an explorer with missing variables is baked. given that we have a solution for csv-based explorers on the data side and bugsnag reporting for indicator-based explorers, I'll downgrade the priority of this issue (but keeping it open to acknowledge that we could do more work here)