Error in cluster_network

[ariannaleoni]

Checklist

• [ x ] I am using the current main branch or the latest release.
• [ x ] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

I encountered a problem when I tried to run the tutorial model of PyPSA-Earth. I tried to debug and it seems that the problem arise on the assert in the cluster_network script: n_clusters >= len(N) and n_clusters <= N.sum() at line 315. n_cluster is equal to 2, while len(N) is equal to 10, so the error occurs. n_cluster is equal to 2 because the number of countries in the model tutorial is two and no bus has a subnetwork assigned, all subnetwork values are empty strings. It seems that the subnetworks are created using " n.determine_network_topology()" at line 480 in cluster_network script, after the initialization of n_cluster with value 2 at line 702 (when cluster == "min"), and no update of the n_cluster is performed. The error that occurred is as follows:

Error Message

Error in rule cluster_network:
    jobid: 4
    input: networks/elec_s.nc, resources/shapes/country_shapes.geojson, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/shapes/gadm_shapes.geojson, resources/costs.csv
    output: networks/elec_s_6.nc, resources/bus_regions/regions_onshore_elec_s_6.geojson, resources/bus_regions/regions_offshore_elec_s_6.geojson, resources/bus_regions/busmap_elec_s_6.csv, resources/bus_regions/linemap_elec_s_6.csv
    log: logs/cluster_network/elec_s_6.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule cluster_network:
    jobid: 4
    input: networks/elec_s.nc, resources/shapes/country_shapes.geojson, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/shapes/gadm_shapes.geojson, resources/costs.csv
    output: networks/elec_s_6.nc, resources/bus_regions/regions_onshore_elec_s_6.geojson, resources/bus_regions/regions_offshore_elec_s_6.geojson, resources/bus_regions/busmap_elec_s_6.csv, resources/bus_regions/linemap_elec_s_6.csv
    log: logs/cluster_network/elec_s_6.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/ariannaleoni/miniconda3/envs/pypsa-cheat/bin/python3.10 /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp3u5evlqh.cluster_network.py' returned non-zero exit status 1.
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/ariannaleoni/miniconda3/envs/pypsa-cheat/bin/python3.10 /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp3u5evlqh.cluster_network.py' returned non-zero exit status 1.
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/concurrent/futures/thread.py", line 58, in run

elec_s_6.log:

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units
WARNING:__main__:Keyword argument feature is only valid for algorithm `hac`. Given feature `solar+onwind-time` will be ignored.
ERROR:_helpers:An error happened in module '/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/scripts/cluster_network.py', function 'distribute_clusters': Number of clusters must be 10 <= n_clusters <= 84 for this selection of countries.
Traceback (most recent call last):
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 758, in <module>
    clustering = clustering_for_n_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 597, in clustering_for_n_clusters
    busmap = busmap_for_n_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 488, in busmap_for_n_clusters
    n_clusters = distribute_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 319, in distribute_clusters
    n_clusters >= len(N) and n_clusters <= N.sum()
AssertionError: Number of clusters must be 10 <= n_clusters <= 84 for this selection of countries.

[ekatef]

Hello @ariannaleoni! Thanks for reporting. You are right that the number of clusters is read from the configuration file and not updated after. That is the expected behaviour, while config.yaml is used to define the parameters which are feasible for simulation.

Could you please share some details to reproduce the error you encounter? Could you please share your config.yaml?

[ariannaleoni]

Hi! Indeed, changing the parameter "cluster" in config.yaml solves the problem. However, there is still an error. Here is the elec_s_10.log file. The config.yaml is identical to the one reported on GitHub, with the only difference being the value of the parameter "cluster," which is now set to 10. In any case, I used the tutorial version, so I assumed I didn't have to change any variables to run a simulation

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units WARNING:main:Keyword argument feature is only valid for algorithm hac. Given feature solar+onwind-time will be ignored. WARNING:main:The configured solver glpk does not support quadratic objectives. Falling back to ipopt. ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py', function 'consense': In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object Traceback (most recent call last): File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 758, in clustering = clustering_for_n_clusters( File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 612, in clustering_for_n_clusters clustering = get_clustering_from_busmap( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 534, in get_clustering_from_busmap buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 341, in aggregatebuses aggregated = n.buses.groupby(busmap).agg(strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1432, in aggregate result = op.agg() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 190, in agg return self.agg_dict_like() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 423, in agg_dict_like return self.agg_or_apply_dict_like(op_name="agg") File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like result_index, result_data = self.compute_dict_like( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 496, in compute_dict_like results = [ File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 497, in getattr(obj._gotitem(key, ndim=1), op_name)(how, kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate return self._python_agg_general(func, *args, *kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general result = self._grouper.agg_series(obj, f) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 863, in agg_series result = self._aggregate_series_pure_python(obj, func) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 884, in _aggregate_series_pure_python res = func(group) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 324, in f = lambda x: func(x, args, kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 136, in consense assert (x == v).all() or x.isnull().all(), ( AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object

[ekatef]

Hi! Indeed, changing the parameter "cluster" in config.yaml solves the problem. However, there is still an error. Here is the elec_s_10.log file. The config.yaml is identical to the one reported on GitHub, with the only difference being the value of the parameter "cluster," which is now set to 10. In any case, I used the tutorial version, so I assumed I didn't have to change any variables to run a simulation

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units WARNING:main:Keyword argument feature is only valid for algorithm hac. Given feature solar+onwind-time will be ignored. WARNING:main:The configured solver glpk does not support quadratic objectives. Falling back to ipopt. ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py', function 'consense': In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object Traceback (most recent call last): File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 758, in clustering = clustering_for_n_clusters( File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 612, in clustering_for_n_clusters clustering = get_clustering_from_busmap( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 534, in get_clustering_from_busmap buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 341, in aggregatebuses aggregated = n.buses.groupby(busmap).agg(strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1432, in aggregate result = op.agg() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 190, in agg return self.agg_dict_like() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 423, in agg_dict_like return self.agg_or_apply_dict_like(op_name="agg") File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like result_index, result_data = self.compute_dict_like( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 496, in compute_dict_like results = [ File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 497, in getattr(obj._gotitem(key, ndim=1), op_name)(how, kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate return self._python_agg_general(func, *args, *kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general result = self._grouper.agg_series(obj, f) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 863, in agg_series result = self._aggregate_series_pure_python(obj, func) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 884, in _aggregate_series_pure_python res = func(group) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 324, in f = lambda x: func(x, args, kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 136, in consense assert (x == v).all() or x.isnull().all(), ( AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object

Hello! That is a weird issue with the mismatch of bus_id. It has been seen before and is linked with environment troubles: it seems to arise if any other dependencies have been installed into pypsa-earth environment. I'd recommend to build a fresh environment, clean-up old outputs of the worflow and run the tutorial again.

[ariannaleoni]

I've already tried to build a fresh environment, but it didn't work. However, I'm using version 3.14.14 of Ipopt because there's no precompiled version for osx-arm64. I've tried to compile the software from source, but I couldn't manage to make it work. However, the differences between my installed version, 3.14.14, and the required one, < 3.13.3, should be backward compatible, that is why I installed the Ipopt 3.14.14. Everything else is a fresh install newly created.

[ekatef]

I've already tried to build a fresh environment, but it didn't work. However, I'm using version 3.14.14 of Ipopt because there's no precompiled version for osx-arm64. I've tried to compile the software from source, but I couldn't manage to make it work. However, the differences between my installed version, 3.14.14, and the required one, < 3.13.3, should be backward compatible, that is why I installed the Ipopt 3.14.14. Everything else is a fresh install newly created.

Ok, thanks for letting us know! It seems your are facing same issue as was reported in #968. The workaround is to remove a constraint to ipop version when creating the environment.

Have you tried to re-run the workflow with a new environment?

[ariannaleoni]

I have already removed the ipopt constraint and re-run the workflow in a new environment, but it didn't work

[ekatef]

I have already removed the ipopt constraint and re-run the workflow in a new environment, but it didn't work

Thank you for trying. Let's try to debug it together. A few troubleshooting questions:

1. Have you also removed outputs of the previous run before re-running the workflow in a fresh environment?

2. Is the error the same: AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree?

3. Could you please share a list of the packages installed in your environment?

[ekatef]

As an additional comment, the issue is unfortunately known but it is tricky to reproduce it. It would be a great achievement, if we would be able to find a way to do so.

[ariannaleoni]

Hello! I've removed the outputs of the previous run before re-running, using the command line snakemake -j 1 clean, and the error is always the same. Anyway, regarding the different sorting of the bus_id, if they are compared as lists (thus with a specific ordering), they are different, but if they are compared as sets (thus the order is no longer relevant), then they are equal. The installed packages in my environment are in this file (they are too many to send as a message) packages environment .docx

[ekatef]

Hello @ariannaleoni!

Thank you for sharing the environment specification. Could it be that something went wrong during creation of the environment? For example, pypsa 0.25.1 was used, while currently there is a restriction <0.25: https://github.com/pypsa-meets-earth/pypsa-earth/blob/413e3fe643a4a9903e145a3853a81cb5944d62a5/envs/environment.yaml#L15

The CI doesn't show any environment problems. I have also generated a fresh environment locally removing a limitation to ipop, and seems to work properly.

If tutorial doesn't work for you, it could make sense to remove the model completely and go through the installation and tutorial following the docs.

[ariannaleoni]

Hi! Apologies for the delayed response, but I've managed to address the issue. Now, I'm attempting to run a new simulation, but I'm encountering some difficulties. I've set the input countries to "africa", but I'm encountering this error: Error in rule build_renewable_profiles: jobid: 17 input: resources/natura.tiff, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_EPSG-4326.tif, data/gebco/GEBCO_2021_TID.nc, resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, data/hydro_capacities.csv, data/eia_hydro_annual_generation.csv, resources/powerplants.csv, resources/bus_regions/regions_onshore.geojson, cutouts/africa-2013-era5.nc output: resources/renewable_profiles/profile_solar.nc log: logs/build_renewable_profile_solar.log (check log file(s) for error details)

After reviewing the log file, I noticed the following message: "WARNING:main:Weather data does not fully cover the requested region." Additionally, I manually downloaded the cutout files "africa-2013-era5" and "cutout-2013-era5", but encountered the same issue. Do you have any suggestions on how to proceed?

[ekatef]

Hi! Apologies for the delayed response, but I've managed to address the issue. Now, I'm attempting to run a new simulation, but I'm encountering some difficulties. I've set the input countries to "africa", but I'm encountering this error: Error in rule build_renewable_profiles: jobid: 17 input: resources/natura.tiff, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_EPSG-4326.tif, data/gebco/GEBCO_2021_TID.nc, resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, data/hydro_capacities.csv, data/eia_hydro_annual_generation.csv, resources/powerplants.csv, resources/bus_regions/regions_onshore.geojson, cutouts/africa-2013-era5.nc output: resources/renewable_profiles/profile_solar.nc log: logs/build_renewable_profile_solar.log (check log file(s) for error details)

After reviewing the log file, I noticed the following message: "WARNING:main:Weather data does not fully cover the requested region." Additionally, I manually downloaded the cutout files "africa-2013-era5" and "cutout-2013-era5", but encountered the same issue. Do you have any suggestions on how to proceed?

Hello @ariannaleoni!

Great, glad to know that the issue has been resolved.

As we discussed, the newly appeared problem has been caused by the fact that the tutorial datasets are created for a particular region only (Nigeria and Benin) and a short timeframe. Since you want to build a full-scale model, the way to go is using parameters of config.default.yaml, adjusting countries parameter.

Regarding the mismatch you observe, I do agree that it should not be the case. You can look into the details of the cutout you downloaded using jupyther notebooks provided into documentation repo. In partucular, build_cutout, build_renewable_profiles and landuse-availability.

[ariannaleoni]

Hello Katia, Thank you for the advice. One possible issue could be that I modified the _simplify_polys function in build_shapes as follows: In the place of return polys.simplify(tolerance=tolerance) (line 257) I inserted this: counter = 0 while counter < 100: try: return polys.simplify(tolerance=tolerance) except Exception as e: counter += 1 logger.info(counter) pass

I had to do it because frequently the old function failed to create the polygons, leading to the code breaking. By doing this, I ensure that the function retries until it succeeds. Could it be a determining factor that causes the problem?

Thank you for the help

[ekatef]

Hello Katia, Thank you for the advice. One possible issue could be that I modified the _simplify_polys function in build_shapes as follows: In the place of return polys.simplify(tolerance=tolerance) (line 257) I inserted this: counter = 0 while counter < 100: try: return polys.simplify(tolerance=tolerance) except Exception as e: counter += 1 logger.info(counter) pass

I had to do it because frequently the old function failed to create the polygons, leading to the code breaking. By doing this, I ensure that the function retries until it succeeds. Could it be a determining factor that causes the problem?

Thank you for the help

Hello @ariannaleoni!

Unfortunately, I'm not quite able to trace the whole chain of possible effects. They may actually be also a result also of some combination of factors. I think, the most effective way to deal with that is to take a step back, return to the original state of the codebase and try to reproduce the error which you experienced in _simplify_polys(). If the issue will be reproducible, a bug report on that would be highly appreciated.

As a comment, if you want to contribute by improving the existing code, that is absolutely welcome! Feel free to look into a list of existing issue and join the developers meetings.

[ariannaleoni]

Hello @ekatef ! If I don't modify _simplify_polis() the error is:

Error in rule build_shapes: jobid: 10 input: data/eez/eez_v11.gpkg output: resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, resources/shapes/africa_shape.geojson, resources/shapes/gadm_shapes.geojson log: logs/build_shapes.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule build_shapes: jobid: 10 input: data/eez/eez_v11.gpkg output: resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, resources/shapes/africa_shape.geojson, resources/shapes/gadm_shapes.geojson log: logs/build_shapes.log (check log file(s) for error details)

RuleException: CalledProcessError in file /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile, line 259: Command 'set -euo pipefail; /Users/ariannaleoni/miniconda3/envs/pypsa-earth/bin/python3.10 /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py' returned non-zero exit status 1. File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile", line 259, in rule_build_shapes File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/concurrent/futures/thread.py", line 58, in run ERROR:snakemake.logging:RuleException: CalledProcessError in file /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile, line 259: Command 'set -euo pipefail; /Users/ariannaleoni/miniconda3/envs/pypsa-earth/bin/python3.10 /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py' returned non-zero exit status 1. File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile", line 259, in rule_build_shapes File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/concurrent/futures/thread.py", line 58, in run

And in the log file:

ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/constructive.py', function 'simplify': IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers Traceback (most recent call last): File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 1342, in offshore_shapes = eez( File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 391, in eez ret_df = ret_df.geometry.map( File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/series.py", line 4691, in map new_values = self._map_values(arg, na_action=na_action) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/base.py", line 919, in _map_values return arr.map(mapper, na_action=na_action) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/arrays/base.py", line 2319, in map return map_array(self, mapper, na_action=na_action) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/algorithms.py", line 1743, in map_array return lib.map_infer(values, mapper, convert=convert) File "lib.pyx", line 2972, in pandas._libs.lib.map_infer File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 392, in lambda x: _simplify_polys(x, minarea=minarea, tolerance=tolerance) File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 261, in _simplify_polys return polys.simplify(tolerance=tolerance) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/geometry/base.py", line 564, in simplify return shapely.simplify(self, tolerance, preserve_topology=preserve_topology) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/decorators.py", line 77, in wrapped return func(*args, kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/constructive.py", line 862, in simplify return lib.simplify_preserve_topology(geometry, tolerance, kwargs) shapely.errors.GEOSException: IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers

Thank you,

Arianna

[ekatef]

shapely.errors.GEOSException: IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers

Hey @ariannaleoni!

Thanks a lot for sharing the details. It looks like for some reason the workflow is not happy with geometry it obtains. Not sure what exactly is the reason, but if the issue is reproducible, it would be amazing to have a fix for it.

Could you please check that the issue can be reproduced and open a bug report for that?

[ariannaleoni]

Hi @ekatef Sure, I'll do that right now