nismod / open-gira

Open-data Global Infrastructure Risk/Resilience Analysis
https://nismod.github.io/open-gira/
MIT License
12 stars 3 forks source link

create_electricity_network.py speed, snkit parallelism #148

Open thomas-fred opened 1 year ago

thomas-fred commented 1 year ago

snkit's link_nodes_to_nearest_edge is called several times and takes a while for large networks. The function doing most of the work is split_edges_at_nodes. SNKIT_PARALLEL can be set to 1 or TRUE to enable parallelism in this function, but it is not constrained (uses os.cpu_count()), so open-gira's snakemake workflow.cores limit is not respected.