Add hierachical clustering and config adjustment

[pz-max]

Describe the feature you'd like to see

PyPSA-Eur recently implemented hierarchical clustering ("hac") options as an alternative to kmeans. The commit gives some hints what scripts need to be adjusted: https://github.com/PyPSA/pypsa-eur/commit/f9a996e5df758f28364d8906baf35ee3f101d7b3 but the implementation should happen according to the PyPSA-Eur repository since multiple changes happened after the original commit: https://github.com/PyPSA/pypsa-eur/blob/44e8fbd5aa846ee47a4b0ee1ae7350cb8fc4da9c/config.default.yaml#L235-L252

Article from @martacki about clustering approaches: "A comparison of clustering methods for the spatial reduction of renewable electricity optimisation models of Europe" https://energyinformatics.springeropen.com/articles/10.1186/s42162-022-00187-7

The to-be-developed config could look like:

cluster_options:
  out_logging: true  # When true, logging is printed to console
  alternative_clustering: false  # "False" use Voronoi shapes, "True" use GADM shapes
  distribute_cluster: ['load'] # ['load'],['pop'] or ['gdp']
  simplify_network:
    algorithm: kmeans # choose from: [hac, kmeans]
    feature: solar+onwind-time # only for hac. choose from: [solar+onwind-time, solar+onwind-cap, solar-time, solar-cap, solar+offwind-cap] etc.
  cluster_network:
    algorithm: kmeans
    feature: solar+onwind-time
  aggregation_strategies:
    generators:
      p_nom_max: sum # use "min" for more conservative assumptions
      p_nom_min: sum
      p_min_pu: mean
      marginal_cost: mean
      committable: any
      ramp_limit_up: max
      ramp_limit_down: max
      efficiency: mean

[martacki]

i'm not sure this is compatible with your repository, but instead of copying lots of code from pypsa-eur, have you ever considered making pypsa-eur a subworkflow of pypsa-meets-africa and importing whole snakemake rules, similar to what's done in the pypsa-breakthrough-energy repository? a rule can then simply call a pypsaeur script, see: https://github.com/PyPSA/pypsa-breakthroughenergy-usa/blob/master/workflow/Snakefile#L220. that would save a lot of work for you guys, basically everything after simplify_network, if the set-up is similar to pypsa-eur (which I haven't checked, but judging based on the Snakefile there might be a big overlap). just a suggestion. that way, hac would be in at the same time the pypsa-eur pr is in.

[pz-max]

@martacki fair point. I add this discussion point to the Agenda of our weekly workflow meeting (every week 14:30-15:00 DE time).

• Do you want to share at the next meeting (or next week) some pros & cons of the sub-workflow?
• I expect that the implementation of this issue will take between 2-4h (if lucky).

[martacki]

which day is your meeting?

[pz-max]

Every Thursday

---

From: Martha Frysztacki @.> Sent: 06 July 2022 17:30 To: pypsa-meets-africa/pypsa-africa @.> Cc: PARZEN Maximilian @.>; Author @.> Subject: Re: [pypsa-meets-africa/pypsa-africa] Add hierachical clustering and config adjustment (Issue #404)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

which day is your meeting?

— Reply to this email directly, view it on GitHubhttps://github.com/pypsa-meets-africa/pypsa-africa/issues/404#issuecomment-1176432406, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOYZENO6WHOHOKHH23RKEUTVSWYC5ANCNFSM52Z5P55Q. You are receiving this because you authored the thread.Message ID: @.***>

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

[ekatef]

I think it should be fixed with #597

[pz-max]

Thank @ekatef :) Let's add a test before closing this. The test should be added here changing kmeans to hac.

[ekatef]

Have tried to add hac to the custom testing config in #692
Currently running testing locally