owid / co2-data

Data on CO2 and greenhouse gas emissions by Our World in Data
https://ourworldindata.org/co2-and-other-greenhouse-gas-emissions
655 stars 219 forks source link

Update cait datasets #24

Closed pabloarosado closed 2 years ago

pabloarosado commented 2 years ago
pabloarosado commented 2 years ago

Thanks @bnjmacdonald, I was trying to follow the philosophy of the energy-data repos. But I agree with you that it would be better to use importers or etl to upload the CAIT datasets to grapher. I could either do it now and cancel this PR, or leave it for a future improvement. I don't have a strong opinion on which option is better, @edomt ?

edomt commented 2 years ago

Indeed the general pipeline isn't optimal – but of course, it was already like that before. There are several things to take into account, including that:

What I would suggest to get rid of the confusion mentioned by @bnjmacdonald is that:

And at the next update of CAIT (presumably next year?), we'll transform the whole thing into a proper etl pipeline.

Let me know what you think :)