Industry module functionality as compared to sector-coupled euro-calliope

[irm-codebase]

Current status of the Industry module

This relates to this branch: https://github.com/calliope-project/ec_modules/tree/feature-industry

It is a perfect match with EC in terms of functionality, but it is far for complete in regards to what SCEC did.

What it does:

• Data-wise:
  • Process eurostat and CHE energy balances and industrial datasets (JRC-IDEES, CHE industry)
  • Produce annual demands for iron and steel, chemicals industry and all other industries (aggregated into one).
  • Allow individual configuration per industry category.
• Model-wise:
  • Assumes certain technology characteristics to translate energy demand and goods demand (tons of steel, etc) into aggregated national annual demand of Diesel, Electricity, Methane, Hydrogen, CO2, etc.

What it does not do

• Data wise:
  • Produce:
    • Disaggregated regional demand (ehighways) is missing for all categories.
      • Industry and chemicals, based on emissions data
      • All other industries, based on workforce and industry freight data
    • Sub national 'weights' for disaggregation combining number of employees (NUTS 2) and freight (NUTS3) data
  • Model-wise:
    • Constraints for industrial demand.

Other data issues

Unfortunately there are other nasty problems that have crept into industry, and should be kept in mind for this module.

• [ ] JRC-IDEES parsing code is too brittle and cannot handle the v2024 dataset. We should rely on ec_jrc_idees for that.
• [ ] Swiss data is fetched directly from the government website. This breaks every year (see https://github.com/calliope-project/euro-calliope/issues/430). Saving it to Zenodo is probably better.
• [ ] We need a separate data processing step for UK industry, similar to CHE, since it is no longer in JRC-IDEES.
• [ ] Initial checks suggest that SCEC did not follow NACE / NST specification when using freight / employment data as proxies. This affects the 'all other industries' group in particular.

Code issues / improvements

• [ ] We should revert back to processing industry as a .csv file. The new netCDF approach introduces a large amount of null values, and is very slow (~15 mins on my laptop). The resulting file goes from 36 MB (all JRC industry files combined) to 2.1 GB!
• [ ] The 'default' env should be updated by removing unnecessary dependencies and moving to modern versions of the dependencies. There are some weird bugs in pandas / netCDF caused by the versions used at the moment.

[irm-codebase]

Looking back at this module, I believe we should be disaggregating into subregions BEFORE we assume how future industry would work.

Arguably, those assumptions should live in Euro-Calliope, since they are too study specific.