Earth-OSM latest version(0.2.0) not compatible

GbotemiB commented 9 months ago

Checklist

[x] I am using the current main branch or the latest release. Please indicate.
[x] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

The current update with earth-osm v0.1.0 is not compatible with the current pypsa-earth installation

This image is from the download_osm_script https://github.com/pypsa-meets-earth/pypsa-earth/blob/main/scripts/download_osm_data.py#L109-L119

This image is from earth-osm repo https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L88-L109

Error Message

Terminal output

ERROR:_helpers:An error happened in module '/Users/gbotemi/Documents/code/PYPSA/mc/pypsa-earth/scripts/download_osm_data.py', function '<module>': get_osm_data() got an unexpected keyword argument 'region_list'
Traceback (most recent call last):
  File "/Users/gbotemi/Documents/code/PYPSA/mc/pypsa-earth/.snakemake/scripts/tmprd57evk9.download_osm_data.py", line 113, in <module>
    eo.get_osm_data(
TypeError: get_osm_data() got an unexpected keyword argument 'region_list'

ekatef commented 9 months ago

Hello @GbotemiB and thanks a lot for reporting! I can reproduce this issue. Currently testing a fix for it.

ekatef commented 9 months ago

Testing with down-graded earth-osm version (earth-osm=0.1.0) resolves the issue with OSM loading but leads to a new one in simplify_network: ValueError( ValueError: unable to infer dtype on variable 'buses_generator'; object array contains mixed native types: str_, str. I fear it may be connected with xarray versions...

@GbotemiB have you found a way to resolve the environment issues?

The full error listing looks like that:

INFO:snakemake.logging:[Tue Dec 19 01:09:28 2023]
rule simplify_network:
    input: networks/AO/elec.nc, resources/AO/costs.csv, resources/AO/bus_regions/regions_onshore.geojson, resources/AO/bus_regions/regions_offshore.geojson
    output: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_onshore_elec_s.geojson, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/busmap_elec_s.csv, resources/AO/bus_regions/connection_costs_s.csv
    log: logs/AO/simplify_network/elec_s.log
    jobid: 7
    benchmark: benchmarks/AO/simplify_network/elec_s
    reason: Missing output files: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/regions_onshore_elec_s.geojson; Input files updated by another job: networks/AO/elec.nc
    wildcards: simpl=
    resources: tmpdir=/var/folders/qn/vpndfm21795ckkq89np1ckp40000gn/T, mem_mb=4000, mem_mib=3815
INFO:snakemake.logging:rule simplify_network:
    input: networks/AO/elec.nc, resources/AO/costs.csv, resources/AO/bus_regions/regions_onshore.geojson, resources/AO/bus_regions/regions_offshore.geojson
    output: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_onshore_elec_s.geojson, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/busmap_elec_s.csv, resources/AO/bus_regions/connection_costs_s.csv
    log: logs/AO/simplify_network/elec_s.log
    jobid: 7
    benchmark: benchmarks/AO/simplify_network/elec_s
    reason: Missing output files: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/regions_onshore_elec_s.geojson; Input files updated by another job: networks/AO/elec.nc
    wildcards: simpl=
    resources: tmpdir=/var/folders/qn/vpndfm21795ckkq89np1ckp40000gn/T, mem_mb=4000, mem_mib=3815

INFO:snakemake.logging:
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
  warnings.warn(
This is the repository path:  /pypsa-earth
Had to go 0 folder(s) up.
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
INFO:pypsa.io:Imported network elec.nc has buses, carriers, generators, lines, loads, storage_units, transformers
Mapping all network lines onto a single 380kV layer
INFO:__main__:Mapping all network lines onto a single 380kV layer
Simplifying connected link components
INFO:__main__:Simplifying connected link components
Removing stubs
INFO:__main__:Removing stubs
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/clustering/spatial.py:104: FutureWarning: The 'axis' keyword in DataFrame.groupby is deprecated and will be removed in a future version.
  new_df = generators.groupby(grouper, axis=0).agg(strategies)
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/clustering/spatial.py:129: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  pnl_df = df_agg.groupby(grouper, axis=1).sum()
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/clustering/spatial.py:200: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  pnl_df = df.groupby(grouper, axis=1).sum()
/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/clustering/spatial.py:200: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  pnl_df = df.groupby(grouper, axis=1).sum()
Dropped 7 buses. A resulted load discrepancy is 2e+01% and 5e+01% for average load and generation capacity, respectively
INFO:__main__:Dropped 7 buses. A resulted load discrepancy is 2e+01% and 5e+01% for average load and generation capacity, respectively
INFO:pypsa.io:Exported network elec_s.nc has generators, buses, loads, storage_units, lines, carriers
ERROR:_helpers:An error happened in module '/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py', function '_infer_dtype': unable to infer dtype on variable 'buses_generator'; object array contains mixed native types: str_, str
Traceback (most recent call last):
  File "/pypsa-earth/.snakemake/scripts/tmpfhxjigcm.simplify_network.py", line 994, in <module>
    n.export_to_netcdf(snakemake.output.network)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/io.py", line 707, in export_to_netcdf
    with ExporterNetCDF(path, compression, float32) as exporter:
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/io.py", line 68, in __exit__
    self.finish()
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/pypsa/io.py", line 402, in finish
    self.ds.to_netcdf(self.path)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/core/dataset.py", line 2310, in to_netcdf
    return to_netcdf(  # type: ignore  # mypy cannot resolve the overloads:(
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/backends/api.py", line 1315, in to_netcdf
    dump_to_store(
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/backends/api.py", line 1362, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/backends/common.py", line 352, in store
    variables, attributes = self.encode(variables, attributes)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/backends/common.py", line 441, in encode
    variables, attributes = cf_encoder(variables, attributes)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py", line 797, in cf_encoder
    new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()}
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py", line 797, in <dictcomp>
    new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()}
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py", line 207, in encode_cf_variable
    var = ensure_dtype_not_object(var, name=name)
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py", line 163, in ensure_dtype_not_object
    data = _copy_with_dtype(data, dtype=_infer_dtype(data, name))
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/site-packages/xarray/conventions.py", line 65, in _infer_dtype
    raise ValueError(
ValueError: unable to infer dtype on variable 'buses_generator'; object array contains mixed native types: str_, str
[Tue Dec 19 01:09:32 2023]
INFO:snakemake.logging:[Tue Dec 19 01:09:32 2023]
Error in rule simplify_network:
    jobid: 7
    input: networks/AO/elec.nc, resources/AO/costs.csv, resources/AO/bus_regions/regions_onshore.geojson, resources/AO/bus_regions/regions_offshore.geojson
    output: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_onshore_elec_s.geojson, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/busmap_elec_s.csv, resources/AO/bus_regions/connection_costs_s.csv
    log: logs/AO/simplify_network/elec_s.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule simplify_network:
    jobid: 7
    input: networks/AO/elec.nc, resources/AO/costs.csv, resources/AO/bus_regions/regions_onshore.geojson, resources/AO/bus_regions/regions_offshore.geojson
    output: networks/AO/elec_s.nc, resources/AO/bus_regions/regions_onshore_elec_s.geojson, resources/AO/bus_regions/regions_offshore_elec_s.geojson, resources/AO/bus_regions/busmap_elec_s.csv, resources/AO/bus_regions/connection_costs_s.csv
    log: logs/AO/simplify_network/elec_s.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /pypsa-earth/Snakefile, line 593:
Command 'set -euo pipefail;  /opt/miniconda3/envs/pypsa-earth-upd20/bin/python3.10 /pypsa-earth/.snakemake/scripts/tmpfhxjigcm.simplify_network.py' returned non-zero exit status 1.
  File "/pypsa-earth/Snakefile", line 593, in __rule_simplify_network
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /pypsa-earth/Snakefile, line 593:
Command 'set -euo pipefail;  /opt/miniconda3/envs/pypsa-earth-upd20/bin/python3.10 /pypsa-earth/.snakemake/scripts/tmpfhxjigcm.simplify_network.py' returned non-zero exit status 1.
  File "/pypsa-earth/Snakefile", line 593, in __rule_simplify_network
  File "/opt/miniconda3/envs/pypsa-earth-upd20/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Removing output files of failed job simplify_network since they might be corrupted:
networks/AO/elec_s.nc, resources/AO/bus_regions/connection_costs_s.csv
WARNING:snakemake.logging:Removing output files of failed job simplify_network since they might be corrupted:
networks/AO/elec_s.nc, resources/AO/bus_regions/connection_costs_s.csv
Shutting down, this might take some time.
WARNING:snakemake.logging:Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
ERROR:snakemake.logging:Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-19T010533.750440.snakemake.log
WARNING:snakemake.logging:Complete log: .snakemake/log/2023-12-19T010533.750440.snakemake.log
(pypsa-earth-upd20) ekaterina@Ekaterinas-MacBook-Pro pypsa-earth %

GbotemiB commented 9 months ago

@ekatef Yeah, I was able to resolve the error by downgrading earth-osm to v0.1.0 and also setting a constraint for xarray to be xarray=2023.11.0. Let me know if this works. I think a quick fix for now would be to set a constraint for earth-osm and xarray.

ekatef commented 9 months ago

@ekatef Yeah, I was able to resolve the error by downgrading earth-osm to v0.1.0 and also setting a constraint for xarray to be xarray=2023.11.0. Let me know if this works. I think a quick fix for now would be to set a constraint for earth-osm and xarray.

Awesome @GbotemiB! I confirm that it works with earth-osm=v0.1.0 and xarray=2023.11.0 🎉 🎉 🎉 Thanks a lot for finding the fix! Do you want to create a PR for that?

GbotemiB commented 9 months ago

Happy to help. I will create a PR and let you know.

davide-f commented 9 months ago

Mmm nice to fix the issue! As a comment, maybe the function has changed name, could you check: https://github.com/pypsa-meets-earth/earth-osm/blob/714ab0d9762d3715fdd1ebb1eba6af1773070004/earth_osm/eo.py#L122

davide-f commented 9 months ago

Note: while tackling this issue, it would be nice to revise the content accounting for the new argument of save_osm_data named "output_dir" this would be our output folder and would help avoid the shutil usage, we could specify as output the output folder in resources (store_path_resources).

The option should help simplify this: https://github.com/pypsa-meets-earth/pypsa-earth/blob/7cf661c1f4ae9e661fce6a9d2dc65aae838f7c24/scripts/download_osm_data.py#L124C4-L141

GbotemiB commented 9 months ago

Mmm nice to fix the issue! As a comment, maybe the function has changed name, could you check: https://github.com/pypsa-meets-earth/earth-osm/blob/714ab0d9762d3715fdd1ebb1eba6af1773070004/earth_osm/eo.py#L122

Oh yeah. The function name changed, but the previous function name was used here https://github.com/pypsa-meets-earth/earth-osm/blob/714ab0d9762d3715fdd1ebb1eba6af1773070004/earth_osm/eo.py#L88-L93

A quick fix will be to just set a constraint for now on earth-osm.

GbotemiB commented 9 months ago

Note: while tackling this issue, it would be nice to revise the content accounting for the new argument of save_osm_data named "output_dir" this would be our output folder and would help avoid the shutil usage, we could specify as output the output folder in resources (store_path_resources).

The option should help simplify this: https://github.com/pypsa-meets-earth/pypsa-earth/blob/7cf661c1f4ae9e661fce6a9d2dc65aae838f7c24/scripts/download_osm_data.py#L124C4-L141

I see what you are talking about. But this current implementation https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L122-L132 is not yet in earth-osm v0.2.0.

GbotemiB commented 9 months ago

Hi all, I tried using the latest version of earth-osm. I also changed the function name in download_osm_data https://github.com/pypsa-meets-earth/pypsa-earth/blob/7cf661c1f4ae9e661fce6a9d2dc65aae838f7c24/scripts/download_osm_data.py#L109-L118 to match the function name in earth-osm https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L122-L132.

I got an error.

INFO:snakemake.logging:16 of 21 steps (76%) done
Select jobs to execute...
WARNING:snakemake.logging:Select jobs to execute...

INFO:snakemake.logging:
[Tue Dec 19 15:05:18 2023]
INFO:snakemake.logging:[Tue Dec 19 15:05:18 2023]
rule cluster_network:
    input: networks/NG/elec_s.nc, resources/NG/shapes/country_shapes.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/shapes/gadm_shapes.geojson, resources/NG/costs.csv
    output: networks/NG/elec_s_6.nc, resources/NG/bus_regions/regions_onshore_elec_s_6.geojson, resources/NG/bus_regions/regions_offshore_elec_s_6.geojson, resources/NG/bus_regions/busmap_elec_s_6.csv, resources/NG/bus_regions/linemap_elec_s_6.csv
    log: logs/NG/cluster_network/elec_s_6.log
    jobid: 4
    benchmark: benchmarks/NG/cluster_network/elec_s_6
    reason: Missing output files: networks/NG/elec_s_6.nc; Input files updated by another job: resources/NG/shapes/gadm_shapes.geojson, resources/NG/shapes/country_shapes.geojson, networks/NG/elec_s.nc, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson
    wildcards: simpl=, clusters=6
    resources: tmpdir=/var/folders/c3/657p__892k72qxh6r8fdkqsh0000gn/T, mem_mb=3000, mem_mib=2862
INFO:snakemake.logging:rule cluster_network:
    input: networks/NG/elec_s.nc, resources/NG/shapes/country_shapes.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/shapes/gadm_shapes.geojson, resources/NG/costs.csv
    output: networks/NG/elec_s_6.nc, resources/NG/bus_regions/regions_onshore_elec_s_6.geojson, resources/NG/bus_regions/regions_offshore_elec_s_6.geojson, resources/NG/bus_regions/busmap_elec_s_6.csv, resources/NG/bus_regions/linemap_elec_s_6.csv
    log: logs/NG/cluster_network/elec_s_6.log
    jobid: 4
    benchmark: benchmarks/NG/cluster_network/elec_s_6
    reason: Missing output files: networks/NG/elec_s_6.nc; Input files updated by another job: resources/NG/shapes/gadm_shapes.geojson, resources/NG/shapes/country_shapes.geojson, networks/NG/elec_s.nc, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson
    wildcards: simpl=, clusters=6
    resources: tmpdir=/var/folders/c3/657p__892k72qxh6r8fdkqsh0000gn/T, mem_mb=3000, mem_mib=2862

INFO:snakemake.logging:
/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
  warnings.warn(
This is the repository path:  /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth
Had to go 0 folder(s) up.
/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/site-packages/pypsa/components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, loads, storage_units
Desired number of clusters (6) higher than the number of buses (1)
ERROR:__main__:Desired number of clusters (6) higher than the number of buses (1)
ERROR:_helpers:An error happened in module '/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/scripts/cluster_network.py', function '<module>': name 'clustering' is not defined
Traceback (most recent call last):
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpo71ol2q6.cluster_network.py", line 779, in <module>
    update_p_nom_max(clustering.network)
NameError: name 'clustering' is not defined
[Tue Dec 19 15:05:21 2023]
INFO:snakemake.logging:[Tue Dec 19 15:05:21 2023]
Error in rule cluster_network:
    jobid: 4
    input: networks/NG/elec_s.nc, resources/NG/shapes/country_shapes.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/shapes/gadm_shapes.geojson, resources/NG/costs.csv
    output: networks/NG/elec_s_6.nc, resources/NG/bus_regions/regions_onshore_elec_s_6.geojson, resources/NG/bus_regions/regions_offshore_elec_s_6.geojson, resources/NG/bus_regions/busmap_elec_s_6.csv, resources/NG/bus_regions/linemap_elec_s_6.csv
    log: logs/NG/cluster_network/elec_s_6.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule cluster_network:
    jobid: 4
    input: networks/NG/elec_s.nc, resources/NG/shapes/country_shapes.geojson, resources/NG/bus_regions/regions_onshore_elec_s.geojson, resources/NG/bus_regions/regions_offshore_elec_s.geojson, resources/NG/shapes/gadm_shapes.geojson, resources/NG/costs.csv
    output: networks/NG/elec_s_6.nc, resources/NG/bus_regions/regions_onshore_elec_s_6.geojson, resources/NG/bus_regions/regions_offshore_elec_s_6.geojson, resources/NG/bus_regions/busmap_elec_s_6.csv, resources/NG/bus_regions/linemap_elec_s_6.csv
    log: logs/NG/cluster_network/elec_s_6.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/gbotemi/miniconda3/envs/pypsa-earth-dev/bin/python3.10 /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpo71ol2q6.cluster_network.py' returned non-zero exit status 1.
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/gbotemi/miniconda3/envs/pypsa-earth-dev/bin/python3.10 /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpo71ol2q6.cluster_network.py' returned non-zero exit status 1.
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-dev/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
WARNING:snakemake.logging:Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
ERROR:snakemake.logging:Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-19T145904.717087.snakemake.log
WARNING:snakemake.logging:Complete log: .snakemake/log/2023-12-19T145904.717087.snakemake.log

ekatef commented 9 months ago

Desired number of clusters (6) higher than the number of buses (1)

Hey @GbotemiB, it looks like there may be something wrong with topology of your network: Desired number of clusters (6) higher than the number of buses (1).

It could be worth to check what is going on in elec.nc and elec_s.nc. I have shared a possible approach for this in this PR.

GbotemiB commented 9 months ago

Thanks @ekatef. I will check it out.

GbotemiB commented 9 months ago

Hi, I used your notebook in this PR.

This was the plots that was generated

Can you draw any conclusion from this plots?

ekatef commented 9 months ago

Hey @GbotemiB, the plots are perfectly alright.

Not sure how does they fit together with the error message... Looking into the error, I had expect that the reason of troubles could be that the whole network has been reduced to a single node. But apparently that is not the case. To me, it looks like some discrepancy in the workflow or a weird environment issue. So, I'd try to clean-up all the workflow files and start again. In case it won't work, some experimenting with environments may be done.

GbotemiB commented 9 months ago

@ekatef, of course, I already tried cleaning up and trying it again. One of the things, I did differently was that I couldn't install ipopt version specified in the environment file. my current version of ipopt is 3.14.5. I don't know if it is related to the cause of this problem.

ekatef commented 9 months ago

@ekatef, of course, I already tried cleaning up and trying it again. One of the things, I did differently was that I couldn't install ipopt version specified in the environment file. my current version of ipopt is 3.14.5. I don't know if it is related to the cause of this problem.

Great! Mmm, theoretically ipopt can cause problems: a solver is used during clustering, as it contains some optimisation. Depending which solvers are available it can be it's exactly ipopt which is utilised. In my case ipopt 3.13.2 for both an older and a fresh environment. So, it could probably make sense to build a fresh environment.

GbotemiB commented 9 months ago

The issue is I unable to install ipopt 3.13.2.

This is my error msg

ekatef commented 9 months ago

The issue is I unable to install ipopt 3.13.2.

This is my error msg

I'd build a new environment using environment.yaml specification. Let's conda search for a suitable version :)

GbotemiB commented 9 months ago

The issue is I unable to install ipopt 3.13.2. This is my error msg

I'd build a new environment using environment.yaml specification. Let's conda search for a suitable version :)

It happens that ipopt is set to use the version below 3.13.3 in the environment.yaml file

I tried removing the constraint. which made it to download 3.14.5 for me.

GbotemiB commented 9 months ago

@ekatef, of course, I already tried cleaning up and trying it again. One of the things, I did differently was that I couldn't install ipopt version specified in the environment file. my current version of ipopt is 3.14.5. I don't know if it is related to the cause of this problem.

Great! Mmm, theoretically ipopt can cause problems: a solver is used during clustering, as it contains some optimisation. Depending which solvers are available it can be it's exactly ipopt which is utilised. In my case ipopt 3.13.2 for both an older and a fresh environment. So, it could probably make sense to build a fresh environment.

Sorry to ask, when you used ipopt 3.12.3, was the version of earth-osm also the latest version. If so, can you try to change the function name here https://github.com/pypsa-meets-earth/pypsa-earth/blob/7cf661c1f4ae9e661fce6a9d2dc65aae838f7c24/scripts/download_osm_data.py#L109-L118 to eo.save_osm_data, then solve the networks.

ekatef commented 9 months ago

The issue is I unable to install ipopt 3.13.2. This is my error msg

I'd build a new environment using environment.yaml specification. Let's conda search for a suitable version :)

It happens that ipopt is set to use the version below 3.13.3 in the environment.yaml file

I tried removing the constraint. which made it to download 3.14.5 for me.

Not sure what was the purpose of the environment modification. Usually version constraints there result from fixing some compatibility issues. So, be careful with them :)

GbotemiB commented 9 months ago

Hmmm. Thanks. I will keep testing with the environment file.

GbotemiB commented 9 months ago

Here is my thought after going through this for a while. I think earth-osm=0.2.0 is the main culprit here. I tried using the latest version of ipopt with earth-osm=0.1.0 which ran successfully.

ekatef commented 9 months ago

Here is my thought after going through this for a while. I think earth-osm=0.2.0 is the main culprit here. I tried using the latest version of ipopt with earth-osm=0.1.0 which ran successfully.

Interesting, thanks for the update. Do I understand properly that applying a restriction to ipopt version leads to troubles with earth-osm=0.2.0?

GbotemiB commented 9 months ago

Here is my thought after going through this for a while. I think earth-osm=0.2.0 is the main culprit here. I tried using the latest version of ipopt with earth-osm=0.1.0 which ran successfully.

Interesting, thanks for the update. Do I understand properly that applying a restriction to ipopt version leads to troubles with earth-osm=0.2.0?

Applying a restriction to ipopt doesnt install on my system. But i dont think it has anything to do with earth-osm=0.2.0. I currently have the latest ipopt version (3.14.13) running in my pypsa-earth environment with no issues.

ekatef commented 9 months ago

Hello @GbotemiB, could you please give an update on this PR and the associated issue? What is currently working and what isn't?

Regarding ipopt version, I see that in PyPSA-Eur it doesn't currently have a version restriction. So, probably compatibility troubles were resolved in meantime, whatever they have been.

GbotemiB commented 9 months ago

Hi @ekatef, the latest version of earth-osm v0.2.0 is not compatible with pypsa-earth due to function name changes in earth-osm=0.2.0. I restrained the version of earth-osm to 0.1.0 and xarray to 2023.11.0 in #939 which current resolves the conflict for an immediate fix.

In the future, another PR can be created to fix the compatibility issues.

ekatef commented 9 months ago

Hi @ekatef, the latest version of earth-osm v0.2.0 is not compatible with pypsa-earth due to function name changes in earth-osm=0.2.0. I restrained the version of earth-osm to 0.1.0 and xarray to 2023.11.0 in #939 which current resolves the conflict for an immediate fix.

In the future, another PR can be created to fix the compatibility issues.

Yeah, agree that version constraints work as a quick fix. However, if I remember properly, Davide has suggested to fix the essence of the problem. Can you do that? :)

GbotemiB commented 9 months ago

Hi @ekatef, the latest version of earth-osm v0.2.0 is not compatible with pypsa-earth due to function name changes in earth-osm=0.2.0. I restrained the version of earth-osm to 0.1.0 and xarray to 2023.11.0 in #939 which current resolves the conflict for an immediate fix. In the future, another PR can be created to fix the compatibility issues.

Yeah, agree that version constraints work as a quick fix. However, if I remember properly, Davide has suggested to fix the essence of the problem. Can you do that? :)

I am still testing it locally, Some of the errors I encountered were what I identified here https://github.com/pypsa-meets-earth/pypsa-earth/issues/936#issuecomment-1863036931

I currently have a new environment to test it on. I will let you know if it works.

GbotemiB commented 9 months ago

Here are my recent findings, what I noticed was that when I switch to earth-osm=0.2.0, I can get everything to work until the clustering point, which gives the error ERROR:__main__:Desired number of clusters (2) higher than the number of buses (1). But when I change the parameter and switch back to earth-osm=0.1.0, everything works normally. By the way, Xarray released a new update recently, which is version 2023.12.0

I am still going to keep checking anyway.

GbotemiB commented 9 months ago

I finally found the issue with earth-osm=0.2.0. The process of downloading data from OSM with earth-osm=0.1.0, names the files with all_raw* and stores it in the data/osm/out before moving it to resources/RDIR/osm/raw.

The new process with earth-osm=0.2.0 is that the files get downloaded into data/osm/out with the filename changed to something like this NG_cable.geojson. This means that the code here won't work https://github.com/pypsa-meets-earth/pypsa-earth/blob/b5d34b4cccd70f846ea9025c8648d0d44c11da93/scripts/download_osm_data.py#L120-L141

I tried to resolve this by rewriting the code locally, then testing it, then I encountered another error

Select jobs to execute...
WARNING:snakemake.logging:Select jobs to execute...

INFO:snakemake.logging:
[Sat Dec 23 04:54:06 2023]
INFO:snakemake.logging:[Sat Dec 23 04:54:06 2023]
rule clean_osm_data:
    input: resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_generators.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/country_shapes.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/shapes/africa_shape.geojson
    output: resources/NG/osm/clean/all_clean_generators.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_substations.geojson
    log: logs/NG/clean_osm_data.log
    jobid: 12
    benchmark: benchmarks/NG/clean_osm_data
    reason: Missing output files: resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_substations.geojson, resources/NG/osm/clean/all_clean_generators.geojson; Updated input files: resources/NG/shapes/country_shapes.geojson, resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/shapes/africa_shape.geojson, resources/NG/osm/raw/all_raw_generators.geojson
    resources: tmpdir=/var/folders/c3/657p__892k72qxh6r8fdkqsh0000gn/T
INFO:snakemake.logging:rule clean_osm_data:
    input: resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_generators.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/country_shapes.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/shapes/africa_shape.geojson
    output: resources/NG/osm/clean/all_clean_generators.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_substations.geojson
    log: logs/NG/clean_osm_data.log
    jobid: 12
    benchmark: benchmarks/NG/clean_osm_data
    reason: Missing output files: resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_substations.geojson, resources/NG/osm/clean/all_clean_generators.geojson; Updated input files: resources/NG/shapes/country_shapes.geojson, resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/shapes/africa_shape.geojson, resources/NG/osm/raw/all_raw_generators.geojson
    resources: tmpdir=/var/folders/c3/657p__892k72qxh6r8fdkqsh0000gn/T

INFO:snakemake.logging:
/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py:823: UserWarning: Geometry is in a geographic CRS. Results from 'buffer' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  .buffer(tolerance)
Process OSM lines
INFO:__main__:Process OSM lines
Add OSM cables to data
INFO:__main__:Add OSM cables to data
/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py:641: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '['3.0' '3.0' '6.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0'
 '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '6.0' '3.0' '3.0' '3.0' '3.0'
 '3.0' '6.0' '6.0' '3.0' '3.0;3.0' '6.0' '6.0' '3.0' '3.0' '3.0' '3.0'
 '3.0' '3.0' '6.0' '6.0' '3.0' '6.0' '6.0' '6.0' '6.0' '6.0' '6.0' '6.0'
 '6.0' '6.0' '6.0' '3.0' '3.0' '6.0' '3.0' '6.0' '6.0' '6.0' '6.0' '6.0'
 '3.0' '3.0;3.0' '3.0' '6.0' '3.0' '3.0' '3.0' '3.0' '6.0' '6.0' '3.0'
 '3.0' '3.0' '6.0' '6.0' '6.0' '6.0' '3.0' '6.0' '3.0' '3.0' '6.0' '6.0'
 '3.0' '3.0' '6.0' '6.0' '6.0' '3.0' '3.0' '6.0' '3.0' '6.0' '3.0' '3.0'
 '6.0' '6.0' '3.0' '3.0' '3.0' '3.0' '6.0' '3.0' '3.0' '6.0' '6.0' '3.0'
 '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0'
 '3.0' '3.0' '6.0' '3.0' '6.0' '6.0' '3.0' '6.0' '6.0' '3.0' '6.0' '3.0'
 '3.0' '3.0' '6.0' '6.0' '6.0' '6.0' '6.0' '3.0' '3.0' '6.0' '6.0' '3.0'
 '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '6.0' '3.0' '3.0' '3.0' '3.0' '3.0'
 '6.0' '3.0' '6.0' '9.0' '6.0' '6.0' '6.0' '6.0' '6.0' '6.0' '3.0' '3.0'
 '3.0' '3.0' '6.0' '3.0' '6.0' '3.0' '3.0;3.0' '3.0' '3.0' '3.0' '3.0'
 '3.0' '3.0' '3.0' '3.0' '6.0' '6.0' '3.0' '3.0' '6.0' '3.0' '3.0' '3.0'
 '3.0' '6.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0' '3.0'
 '6.0' '3.0' '3.0' '3.0']' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  df.loc[filled_values.index, "circuits"] = filled_values
/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py:669: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
  is_all_list = df[cols].applymap(lambda x: isinstance(x, list)).all(axis=1)
/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py:678: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
  nunique_values = df[cols].applymap(len).nunique(axis=1)
Filter lines by voltage, frequency, circuits and geometry
INFO:__main__:Filter lines by voltage, frequency, circuits and geometry
Select lines and cables in the region of interest
INFO:__main__:Select lines and cables in the region of interest
Setting lines country name using the GADM shapes
INFO:__main__:Setting lines country name using the GADM shapes
Saving lines output
INFO:__main__:Saving lines output
Process OSM substations
INFO:__main__:Process OSM substations
ERROR:_helpers:An error happened in module '/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/site-packages/geopandas/array.py', function 'x': x attribute access only provided for Point geometries
Traceback (most recent call last):
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py", line 1105, in <module>
    clean_data(
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py", line 973, in clean_data
    df_all_substations = prepare_substation_df(df_all_substations)
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py", line 56, in prepare_substation_df
    df_all_substations["lon"] = df_all_substations["geometry"].x
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/site-packages/geopandas/geoseries.py", line 258, in x
    return _delegate_property("x", self)
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/site-packages/geopandas/base.py", line 70, in _delegate_property
    data = getattr(a_this, op)
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/site-packages/geopandas/array.py", line 972, in x
    raise ValueError(message)
ValueError: x attribute access only provided for Point geometries
[Sat Dec 23 04:54:08 2023]
INFO:snakemake.logging:[Sat Dec 23 04:54:08 2023]
Error in rule clean_osm_data:
    jobid: 12
    input: resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_generators.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/country_shapes.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/shapes/africa_shape.geojson
    output: resources/NG/osm/clean/all_clean_generators.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_substations.geojson
    log: logs/NG/clean_osm_data.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule clean_osm_data:
    jobid: 12
    input: resources/NG/osm/raw/all_raw_cables.geojson, resources/NG/osm/raw/all_raw_generators.geojson, resources/NG/osm/raw/all_raw_lines.geojson, resources/NG/osm/raw/all_raw_substations.geojson, resources/NG/shapes/country_shapes.geojson, resources/NG/shapes/offshore_shapes.geojson, resources/NG/shapes/africa_shape.geojson
    output: resources/NG/osm/clean/all_clean_generators.geojson, resources/NG/osm/clean/all_clean_generators.csv, resources/NG/osm/clean/all_clean_lines.geojson, resources/NG/osm/clean/all_clean_substations.geojson
    log: logs/NG/clean_osm_data.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile, line 206:
Command 'set -euo pipefail;  /Users/gbotemi/miniconda3/envs/pypsa-earth-test/bin/python3.10 /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py' returned non-zero exit status 1.
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile", line 206, in __rule_clean_osm_data
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile, line 206:
Command 'set -euo pipefail;  /Users/gbotemi/miniconda3/envs/pypsa-earth-test/bin/python3.10 /Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/.snakemake/scripts/tmpgugxt0t4.clean_osm_data.py' returned non-zero exit status 1.
  File "/Users/gbotemi/Documents/code/PYPSA/bug/pypsa-earth/Snakefile", line 206, in __rule_clean_osm_data
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-test/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Removing output files of failed job clean_osm_data since they might be corrupted:
resources/NG/osm/clean/all_clean_lines.geojson
WARNING:snakemake.logging:Removing output files of failed job clean_osm_data since they might be corrupted:
resources/NG/osm/clean/all_clean_lines.geojson
Shutting down, this might take some time.
WARNING:snakemake.logging:Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
ERROR:snakemake.logging:Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-23T045404.748603.snakemake.log
WARNING:snakemake.logging:Complete log: .snakemake/log/2023-12-23T045404.748603.snakemake.log

I was able to trace the error to ValueError: x attribute access only provided for Point geometries, which seems to be a result of the files downloaded from with earth-osm=0.2.0

ekatef commented 9 months ago

I finally found the issue with earth-osm=0.2.0. The process of downloading data from OSM with earth-osm=0.1.0, names the files with all_raw* and stores it in the data/osm/out before moving it to resources/RDIR/osm/raw.

The new process with earth-osm=0.2.0 is that the files get downloaded into data/osm/out with the filename changed to something like this NG_cable.geojson. This means that the code here won't work

https://github.com/pypsa-meets-earth/pypsa-earth/blob/b5d34b4cccd70f846ea9025c8648d0d44c11da93/scripts/download_osm_data.py#L120-L141

Hey @GbotemiB, great result! Feel free to fix the folder names in your PR.

GbotemiB commented 9 months ago

I finally found the issue with earth-osm=0.2.0. The process of downloading data from OSM with earth-osm=0.1.0, names the files with all_raw* and stores it in the data/osm/out before moving it to resources/RDIR/osm/raw. The new process with earth-osm=0.2.0 is that the files get downloaded into data/osm/out with the filename changed to something like this NG_cable.geojson. This means that the code here won't work https://github.com/pypsa-meets-earth/pypsa-earth/blob/b5d34b4cccd70f846ea9025c8648d0d44c11da93/scripts/download_osm_data.py#L120-L141

Hey @GbotemiB, great result! Feel free to fix the folder names in your PR.

I worked on it already, but it led to another error ValueError: x attribute access only provided for Point geometries. To verify the culprit, I replaced the files in resources/RDIR/osm/raw with the files downloaded with earth-osm=0.1.0 which worked seamlessly. Looks like there are a lot of changes with earth-osm=0.2.0 workflow.

davide-f commented 9 months ago

Indeed, let's use the simple fix for the moment. An issue has been opened on earth-osm to notify @mnm-matin . For the moment we can fix the version to 0.1 :)

pypsa-meets-earth / pypsa-earth