feat(explorers): allow referencing indicators by ETL path

So!

I worked on using ETL paths in indicator-based explorers for the last few days. This means that something like this is now valid: CleanShot 2024-06-26 at 12 11 26

How this works

The way this works is that at baking (or preview) time, we create a map like so

const catalogPathToIndicatorIdMap = 
//EMBEDDED_EXPLORER_CATALOG_PATH_TO_INDICATOR_ID_MAP
{
  "grapher/demography/2023-03-31/population/population#population": 597929,
  "grapher/demography/2023-03-31/population/population#population_historical": 597930,
  // ...
}
//EMBEDDED_EXPLORER_CATALOG_PATH_TO_INDICATOR_ID_MAP
;

This is alongside the existing grapherConfigs and partialGrapherConfigs objects that serve a similar purpose.

Then, at render time, the explorer resolves any non-numeric values contained in the fields yVariableIds, xVariableId, sizeVariableId, colorVariableId, as well as the column def's variableId, using this mapping.

Basically, this means that the client still knows about the ETL paths used, but it also knows how to resolve them to indicator IDs (which it, of course, knows how to fetch).

If any of the ETL paths are invalid, a Bugsnag error will be sent at bake time. The explorer will still get baked, but will be partially-broken.

As you saw above, the fields have not been renamed. That is, any of yVariableIds etc. can now contain either an integer variable ID or a string catalog path (or, in the case of yVariableIds, in theory also a mix of both).

Other alternatives considered

I briefly considered to apply the catalog path -> indicator ID conversion completely at bake time, meaning that the client-side explorer code would have no need at all to be aware of catalog paths, and catalogPathToIndicatorIdMap wouldn't be necessary at all: The explorer config shipped to the client simply wouldn't include any catalog paths any more.

However, this would've meant transforming the explorer config during the baking process, which we currently don't have any way to do, and the config format isn't necessarily great for doing that. Given that, I decided the potential upside wouldn't be worth the extra effort.

Todos

[ ] Should we also produce errors for any invalid slugs contained in column def variableId?
[ ] Do we also send Bugsnag errors when previewing?
[ ] Fix test failure (maybe get rid of test?)
[ ] Check that embedding these explorers works fine
[ ] Try to rewrite explorer program with ETL path resolution already at bake time (timeboxed 2h)
[ ] Think about having a explorer-global catalogPathPrefix or so?

owid / owid-grapher