owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
86 stars 23 forks source link

`etl run --export` should default to a matching branch for other repos #3537

Open larsyencken opened 1 week ago

larsyencken commented 1 week ago

Problem

In our ETL staging work, we are moving to using matching branches (my-data-change) on both the ETL and owid-content.

Currently our etl run --export does a dry-run push to master on owid-content and other repositories, such as co2-data. You use an environment variable to pick which branch to do a real push to. But actually we should default to the matching branch name in the external repo.

This will make them work more harmoniously with the new staging flow.

larsyencken commented 1 week ago

Recall:

Marigold commented 12 hours ago

This has been implemented for etlr github/co2_data --export step in this PR https://github.com/owid/etl/pull/3605. Other steps could use the same pattern.