:tada: Create staging server with ETL and Admin for new PRs

[Marigold]

Staging servers are automatically created for all new PRs in owid-grapher repository with Admin and Cloudflare Pages in dev mode. We learned to use them for ETL workflow, too - most of us have a staging server with their name (like staging-site-mojmir) where we push datasets, play with charts and share them with others.

Taking this even further, we could spin up an automated staging server with ETL and Admin installed. It would run etl ... --grapher on new commits, and so, in theory, we wouldn't even have to run them locally. This could save dev time, especially for large datasets.

Notes

• ETL catalog shouldn't be built from scratch - we should copy data/ folder from master and only run changed steps (we already do it for unit tests)
• This is pretty similar to the buildkite job that runs ETL in CI... perhaps we can unify them?
  • One problem is that spinning up Admin takes ~20min, so it could slow things down
• What to do if there are two branches in etl and owid-grapher with the same name?

Transferring changes from the staging server to live

If we create / migrate charts on the staging server, it'd be nice to have a way to transfer those changes to live to avoid recreating the same charts. There might be some technical difficulties, but hopefully nothing major.

Issues

• The staging server might use different variable IDs, how to map them? What about chart configs?
• In order to transfer changes, we have to merge the PR first to build the dataset in live. Closing PR would generally terminate the staging server

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Marigold]

No way, we have this!