quintel / etlocal

Regional data analysis tool for the Energy Transition Model
https://data.energytransitionmodel.com/
MIT License
3 stars 2 forks source link

Default description of EB() method no longer correct for all European country datasets #557

Open kndehaan opened 3 weeks ago

kndehaan commented 3 weeks ago

Various keys in ETLocal are calculated with the EB() method where data is obtained from the energy balance. The default commit message used for these keys is "Value sourced from the Eurostat energy balance", however it is not always the case that the energy balance is obtained from Eurostat. Therefore, the default description should be changed.

I see two options:

  1. Only change the default commit message, making it more generic, such as "Value sourced from country energy balance". Downside of this is that currently, a link to the Eurostat energy balance is provided in the default commit message, making it transparant where data comes from in case of Eurostat energy balances. When using a generic description, it would not be logic to refer to the Eurostat energy balance anymore, herewith decreasing transparancy.
  2. Use a new attribute when creating country dataset (with dataset_source set to entso) indicating whether an energy balance is sourced from Eurostat, e.g. indicating in the data.csv eurostat_eb is True. If False, generic descriptions for the commit messages can be applied. This requires adding this new attribute to all country datasets that use the EB() method, as well as changes in the setup of the local files and add generic description for non-eurostat EB.

@mabijkerk @kaskranenburgQ let me know what you think.

mabijkerk commented 6 days ago

Because we appended the EB filenames with "energy_balance_enriched" we need to update the links anyway. Perhaps this is also what @louispt1 is referring to in https://github.com/quintel/etlocal/issues/559.

I'm fine with option 1 to make the commit message more generic. Ideally, we would refer to the dataset pipeline instead of the files on ETLocal. The files on ETLocal are only for ETLocal to be able to read the data, so this doesn't give users a lot of additional insights. Since only CH, NO and RS have nicely documented pipelines on ETDataset now, this is something we might do in the future (and we can discuss it in the "Streamlining datasets" project). Let's keep it simple for now!

Finally, I'd recommend renaming the attribute in the interface_elements that specifies that a key should be read from the EB. It is now marked with entso, which does not make sense. I would rename it to energy_balance to facilitate both Europe and World EBs.