search

PyPSA / pypsa-usa

PyPSA-USA: An Open-Source Energy System Optimization Model for the United States
https://pypsa-usa.readthedocs.io
MIT License
49 stars 19 forks source link

`atlite` throws in error in `build_renewable_profiles` #106

Closed degleris1 closed 10 months ago

degleris1 commented 11 months ago

Checklist

The Issue

@ktehranchi There seems to be an issue either with dask, atlite, or the call to atlite in build_renewable_profiles.py. Specifically, the issue seems to be occurring during the call to func on line 296:

func = getattr(cutout, resource.pop("method"))
    if client is not None:
        resource["dask_kwargs"] = {"scheduler": client}
    capacity_factor = correction_factor * func(capacity_factor=True, **resource)
    layout = capacity_factor * area * capacity_per_sqkm
    profile, capacities = func(
        matrix=availability.stack(spatial=["y", "x"]),
        layout=layout,
        index=buses,
        per_unit=True,
        return_capacity=True,
        **resource,
    )

This was the initial bug @nomineb ran into a week ago or so. I posted my conda list below, but I am running atlite 0.2.11 and dask 2023.10.0. 

INFO:__main__:correction_factor is set as 0.8855
/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/xarray/core/dataset.py:270: UserWarning: The specified chunks separate the stored chunks along dimension "time" starting at index 100. This could degrade performance. Instead, consider rechunking after loading.
  warnings.warn(
INFO:__main__:Calculate landuse availabilities...
INFO:__main__:Completed availability calculation (4.66s)
INFO:atlite.convert:Convert and aggregate 'wind'.
INFO:atlite.convert:Convert and aggregate 'wind'.
Traceback (most recent call last):
  File "/Users/degleris/pypsa-usa/workflow/.snakemake/scripts/tmpxym2h1tj.build_renewable_profiles.py", line 300, in <module>
    profile, capacities = func(
                          ^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/atlite/convert.py", line 521, in wind
    return cutout.convert_and_aggregate(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/atlite/convert.py", line 174, in convert_and_aggregate
    results = aggregate_matrix(da, matrix=matrix, index=index)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/atlite/aggregate.py", line 19, in aggregate_matrix
    return xr.apply_ufunc(
           ^^^^^^^^^^^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/xarray/core/computation.py", line 1249, in apply_ufunc
    return apply_dataarray_vfunc(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/xarray/core/computation.py", line 308, in apply_dataarray_vfunc
    result_var = func(*data_vars)
                 ^^^^^^^^^^^^^^^^
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/site-packages/xarray/core/computation.py", line 754, in apply_variable_ufunc
    raise ValueError(
ValueError: dimension spatial on 0th function argument to apply_ufunc with dask='parallelized' consists of multiple chunks, but is also a core dimension. To fix, either rechunk into a single array chunk along this dimension, i.e., ``.chunk(dict(spatial=-1))``, or pass ``allow_rechunk=True`` in ``dask_gufunc_kwargs`` but beware that this may significantly increase memory usage.
[Tue Oct 17 18:22:53 2023]
Error in rule build_renewable_profiles:
    jobid: 15
    input: resources/western/elec_base_network.nc, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_USA_EPSG-4326.tif, data/natura.tiff, data/gebco/gebco_2023_tid_USA.nc, resources/western/country_shapes.geojson, resources/western/offshore_shapes.geojson, resources/western/regions_offshore.geojson, cutouts/western_era5_2019.nc
    output: resources/western/profile_offwind.nc
    log: logs/western/build_renewable_profile_offwind.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/degleris/pypsa-usa/workflow/Snakefile, line 409:
Command 'set -euo pipefail;  /Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/bin/python3.11 /Users/degleris/pypsa-usa/workflow/.snakemake/scripts/tmpxym2h1tj.build_renewable_profiles.py' returned non-zero exit status 1.
  File "/Users/degleris/pypsa-usa/workflow/Snakefile", line 409, in __rule_build_renewable_profiles
  File "/Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2/lib/python3.11/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-17T182008.346661.snakemake.log

Steps To Reproduce

Downloaded from scratch. Used conda env create -f environment.yaml. Added databundles manually since Zenodo isn't working. Ran

snakemake -j 4

and also tried snakemake -j 1.

Expected Behavior

No response

Error Message

No response

Anything else?

Conda environment:

# packages in environment at /Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2:
#
# Name                    Version                   Build  Channel
affine                    2.4.0              pyhd8ed1ab_0    conda-forge
amply                     0.1.6              pyhd8ed1ab_0    conda-forge
anyio                     4.0.0              pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py311h2725bcf_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.4.0              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
atlite                    0.2.11             pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
aws-c-auth                0.7.4                hbe6ead2_2    conda-forge
aws-c-cal                 0.6.2                hfc10710_2    conda-forge
aws-c-common              0.9.3                h0dc2134_0    conda-forge
aws-c-compression         0.2.17               hd41bdd4_3    conda-forge
aws-c-event-stream        0.3.2                he52df9d_2    conda-forge
aws-c-http                0.7.13               hb43ddd7_2    conda-forge
aws-c-io                  0.13.33              h2566903_0    conda-forge
aws-c-mqtt                0.9.7                h6b10715_0    conda-forge
aws-c-s3                  0.3.17               h5800b94_4    conda-forge
aws-c-sdkutils            0.1.12               hd41bdd4_2    conda-forge
aws-checksums             0.1.17               hd41bdd4_2    conda-forge
aws-crt-cpp               0.24.2               hf47b73b_2    conda-forge
aws-sdk-cpp               1.11.156             hf61f2bb_4    conda-forge
babel                     2.13.0             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.5               heccf04b_0    conda-forge
bokeh                     3.3.0              pyhd8ed1ab_0    conda-forge
bottleneck                1.3.7           py311h4a70a88_1    conda-forge
branca                    0.6.0              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                h0dc2134_1    conda-forge
brotli-bin                1.1.0                h0dc2134_1    conda-forge
brotli-python             1.1.0           py311hdf8f085_1    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.20.1               h10d778d_0    conda-forge
c-blosc2                  2.10.5               h354e526_0    conda-forge
ca-certificates           2023.7.22            h8857fd0_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.18.0               h99e66fa_0    conda-forge
cartopy                   0.22.0          py311hab14417_0    conda-forge
cdsapi                    0.6.1              pyhd8ed1ab_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py311hc0b63fd_0    conda-forge
cfitsio                   4.3.0                h66f91ea_0    conda-forge
cftime                    1.6.2           py311h4a70a88_2    conda-forge
charset-normalizer        3.3.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
coin-or-cbc               2.10.10              he49632c_0    conda-forge
coin-or-cgl               0.60.7               ha3c4b8c_0    conda-forge
coin-or-clp               1.17.8               hf0ee74e_0    conda-forge
coin-or-osi               0.108.8              hfef9e4d_0    conda-forge
coin-or-utils             2.11.9               he494582_0    conda-forge
coincbc                   2.10.10           0_metapackage    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
configargparse            1.7                pyhd8ed1ab_0    conda-forge
connection_pool           0.0.3              pyhd3deb0d_0    conda-forge
contourpy                 1.1.1           py311h5fe6e05_1    conda-forge
country_converter         1.0.0              pyhd8ed1ab_1    conda-forge
countrycode               0.4.0                    pypi_0    pypi
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cytoolz                   0.12.2          py311h2725bcf_1    conda-forge
dask                      2023.10.0          pyhd8ed1ab_0    conda-forge
dask-core                 2023.10.0          pyhd8ed1ab_0    conda-forge
datrie                    0.8.2           py311h2725bcf_7    conda-forge
debugpy                   1.8.0           py311hdf8f085_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecation               2.1.0              pyh9f0ad1d_0    conda-forge
descartes                 1.1.0                      py_4    conda-forge
distributed               2023.10.0          pyhd8ed1ab_0    conda-forge
docutils                  0.20.1          py311h6eed73b_2    conda-forge
dpath                     2.1.6              pyha770c72_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hf0c8a7f_1    conda-forge
fiona                     1.9.5           py311hf14a637_0    conda-forge
folium                    0.14.0             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h5bb23bf_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311he705e18_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               h60636b9_2    conda-forge
freexl                    2.0.0                h3ec172f_0    conda-forge
fs                        2.4.11             pyhd8ed1ab_3    conda-forge
fsspec                    2023.9.2           pyh1a96a4e_0    conda-forge
gdal                      3.7.2           py311h5646c56_7    conda-forge
geographiclib             1.52               pyhd8ed1ab_0    conda-forge
geopandas                 0.14.0             pyhd8ed1ab_1    conda-forge
geopandas-base            0.14.0             pyha770c72_1    conda-forge
geopy                     2.4.0              pyhd8ed1ab_0    conda-forge
geos                      3.12.0               he965462_0    conda-forge
geotiff                   1.7.1               h889ec99_14    conda-forge
gettext                   0.21.1               h8a4c099_0    conda-forge
gflags                    2.2.2             hb1e8313_1004    conda-forge
giflib                    5.2.1                hb7f2c08_3    conda-forge
gitdb                     4.0.10             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.38             pyhd8ed1ab_0    conda-forge
glog                      0.6.0                h8ac2a54_0    conda-forge
gurobipy                  10.0.3                   pypi_0    pypi
hdf4                      4.2.15               h8138101_7    conda-forge
hdf5                      1.14.2          nompi_hedada53_100    conda-forge
highspy                   1.5.3                    pypi_0    pypi
humanfriendly             10.0               pyhd8ed1ab_6    conda-forge
icu                       73.2                 hf5e326d_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.1.0              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.25.2             pyh1050b4e_0    conda-forge
ipython                   8.16.1             pyh31c8845_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                8.1.1              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
json-c                    0.17                 h8e11ae5_0    conda-forge
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py311h6eed73b_3    conda-forge
jsonschema                4.19.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.19.1             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0             pyhd8ed1ab_10    conda-forge
jupyter-lsp               2.2.0              pyhd8ed1ab_0    conda-forge
jupyter_client            8.4.0              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.4.0           py311h6eed73b_0    conda-forge
jupyter_events            0.8.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.8.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab                4.0.7              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.25.0             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.9              pyhd8ed1ab_0    conda-forge
kealib                    1.5.2                h052fcf7_1    conda-forge
kiwisolver                1.4.5           py311h5fe6e05_1    conda-forge
krb5                      1.21.2               hb884880_0    conda-forge
lcms2                     2.15                 hd6ba6f3_3    conda-forge
lerc                      4.0.0                hb486fe8_0    conda-forge
libabseil                 20230802.1      cxx17_h048a20a_0    conda-forge
libaec                    1.1.2                he965462_1    conda-forge
libarchive                3.7.2                h0b5dc4a_0    conda-forge
libarrow                  13.0.0           h73e7a6f_9_cpu    conda-forge
libblas                   3.9.0           19_osx64_openblas    conda-forge
libboost-headers          1.82.0               h694c41f_6    conda-forge
libbrotlicommon           1.1.0                h0dc2134_1    conda-forge
libbrotlidec              1.1.0                h0dc2134_1    conda-forge
libbrotlienc              1.1.0                h0dc2134_1    conda-forge
libcblas                  3.9.0           19_osx64_openblas    conda-forge
libcrc32c                 1.1.2                he49afe7_0    conda-forge
libcurl                   8.4.0                h726d00d_0    conda-forge
libcxx                    16.0.6               hd57cbcb_0    conda-forge
libdeflate                1.19                 ha4e1b8e_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libevent                  2.1.12               ha90c15b_1    conda-forge
libexpat                  2.5.0                hf0c8a7f_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgdal                   3.7.2                h926149b_7    conda-forge
libgfortran               5.0.0           13_2_0_h97931a8_1    conda-forge
libgfortran5              13.2.0               h2873a65_1    conda-forge
libglib                   2.78.0               hc62aa5d_0    conda-forge
libgoogle-cloud           2.12.0               h407922f_3    conda-forge
libgrpc                   1.58.1               hecc90c7_2    conda-forge
libiconv                  1.17                 hac89ed1_0    conda-forge
libjpeg-turbo             3.0.0                h0dc2134_1    conda-forge
libkml                    1.3.0             hab3ca0e_1018    conda-forge
liblapack                 3.9.0           19_osx64_openblas    conda-forge
liblapacke                3.9.0           19_osx64_openblas    conda-forge
libnetcdf                 4.9.2           nompi_h6a32802_112    conda-forge
libnghttp2                1.52.0               he2ab024_0    conda-forge
libopenblas               0.3.24          openmp_h48a4ad5_0    conda-forge
libpng                    1.6.39               ha978bb4_0    conda-forge
libpq                     16.0                 h3df487d_1    conda-forge
libprotobuf               4.24.3               he0c2237_1    conda-forge
libre2-11                 2023.06.02           h4694dbf_0    conda-forge
librttopo                 1.1.0               h23f359d_14    conda-forge
libsodium                 1.0.18               hbcb3906_1    conda-forge
libspatialindex           1.9.3                he49afe7_4    conda-forge
libspatialite             5.1.0                h231fb02_0    conda-forge
libsqlite                 3.43.2               h92b6c6a_0    conda-forge
libssh2                   1.11.0               hd019ec5_0    conda-forge
libthrift                 0.19.0               h064b379_1    conda-forge
libtiff                   4.6.0                h684deea_2    conda-forge
libutf8proc               2.8.0                hb7f2c08_0    conda-forge
libwebp-base              1.3.2                h0dc2134_0    conda-forge
libxcb                    1.15                 hb7f2c08_0    conda-forge
libxml2                   2.11.5               h3346baf_1    conda-forge
libxslt                   1.1.37               h20bfa82_1    conda-forge
libzip                    1.10.1               hc158999_3    conda-forge
libzlib                   1.2.13               h8a1eda9_5    conda-forge
linopy                    0.2.6              pyhd8ed1ab_0    conda-forge
llvm-openmp               17.0.2               hff08bdf_0    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lxml                      4.9.3           py311h19a211c_1    conda-forge
lz4                       4.3.2           py311hbaf8d61_1    conda-forge
lz4-c                     1.9.4                hf0c8a7f_0    conda-forge
lzo                       2.10              haf1e3a3_1000    conda-forge
mapclassify               2.6.1              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py311h2725bcf_1    conda-forge
matplotlib                3.8.0           py311h6eed73b_2    conda-forge
matplotlib-base           3.8.0           py311hd316c10_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
memory_profiler           0.61.0             pyhd8ed1ab_0    conda-forge
minizip                   4.0.1                h64dbd51_5    conda-forge
mistune                   3.0.1              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.6           py311h5fe6e05_0    conda-forge
munch                     4.0.0              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.9.2              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.9.2              pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.9.2              pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  hf0c8a7f_0    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
netcdf4                   1.6.4           nompi_py311h710f573_103    conda-forge
networkx                  3.1                pyhd8ed1ab_0    conda-forge
notebook                  7.0.6              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 hea0b92c_0    conda-forge
nss                       3.94                 hd6ac835_0    conda-forge
numexpr                   2.8.7           py311h1eadf79_3    conda-forge
numpy                     1.26.0          py311hc44ba51_0    conda-forge
openjdk                   21.0.0               hf4d7fad_0    conda-forge
openjpeg                  2.5.0                ha4da562_3    conda-forge
openpyxl                  3.1.2           py311h2725bcf_1    conda-forge
openssl                   3.1.3                h8a1eda9_0    conda-forge
orc                       1.9.0                hb037d9a_3    conda-forge
overrides                 7.4.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.0.3           py311hab14417_1    conda-forge
pandoc                    3.1.3                h9d075a6_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1              pyhd8ed1ab_0    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                h1c4e4bc_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311hea5c87a_0    conda-forge
pip                       23.3               pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               he965462_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
plac                      1.4.0              pyhd8ed1ab_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
poppler                   23.10.0              hdd5a5e8_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                16.0                 hc940a54_1    conda-forge
progressbar2              4.2.0              pyhd8ed1ab_0    conda-forge
proj                      9.3.0                h7d51f5a_1    conda-forge
prometheus_client         0.17.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h2725bcf_1    conda-forge
pthread-stubs             0.4               hc929b4f_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pulp                      2.7.0           py311h6eed73b_1    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
pyarrow                   13.0.0          py311h54e7ce8_9_cpu    conda-forge
pycountry                 22.3.5             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pyobjc-core               10.0            py311hf110eff_0    conda-forge
pyobjc-framework-cocoa    10.0            py311hf110eff_1    conda-forge
pyomo                     6.6.2           py311hdf8f085_1    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pyproj                    3.6.1           py311h7851242_2    conda-forge
pypsa                     0.25.1             pyhd8ed1ab_0    conda-forge
pyshp                     2.3.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytables                  3.9.1           py311h3cee394_0    conda-forge
pytest                    7.4.2              pyhd8ed1ab_0    conda-forge
python                    3.11.6          h30d4d87_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python-utils              3.8.1              pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py311h2725bcf_1    conda-forge
pyzmq                     25.1.1          py311h5dacc12_1    conda-forge
qtconsole-base            5.4.4              pyha770c72_0    conda-forge
qtpy                      2.4.0              pyhd8ed1ab_0    conda-forge
rasterio                  1.3.8           py311hbe066b3_4    conda-forge
re2                       2023.06.02           hd34609a_0    conda-forge
readline                  8.2                  h9e318b2_1    conda-forge
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
reretry                   0.11.8             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rpds-py                   0.10.6          py311h5e0f0e4_0    conda-forge
rtree                     1.1.0           py311hbc1f44b_0    conda-forge
scikit-learn              1.3.1           py311h66081b9_1    conda-forge
scipy                     1.11.3          py311h16c3c4d_1    conda-forge
seaborn                   0.13.0               hd8ed1ab_0    conda-forge
seaborn-base              0.13.0             pyhd8ed1ab_0    conda-forge
send2trash                1.8.2              pyhd1c38e8_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
setuptools-scm            8.0.4              pyhd8ed1ab_0    conda-forge
setuptools_scm            8.0.4                hd8ed1ab_0    conda-forge
shapely                   2.0.2           py311h359915d_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smart_open                6.4.0              pyhd8ed1ab_0    conda-forge
smmap                     3.0.5              pyh44b312d_0    conda-forge
snakemake-minimal         7.32.4             pyhdfd78af_1    bioconda
snappy                    1.1.10               h225ccf5_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sqlite                    3.43.2               h7461747_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.0          py311hc9a392d_2    conda-forge
stopit                    1.1.2                      py_0    conda-forge
tabula-py                 2.7.0           py311h6eed73b_1    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tblib                     2.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.17.1             pyhd1c38e8_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
throttler                 1.2.2              pyhd8ed1ab_0    conda-forge
tiledb                    2.16.3               hd3a41d5_3    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               hef22860_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
toposort                  1.10               pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h2725bcf_1    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.11.2             pyhd8ed1ab_0    conda-forge
tsam                      2.3.1                    pypi_0    pypi
types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzcode                    2023c                hb7f2c08_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
uriparser                 0.9.7                hf0c8a7f_1    conda-forge
urllib3                   2.0.6              pyhd8ed1ab_0    conda-forge
validators                0.22.0             pyhd8ed1ab_0    conda-forge
vresutils                 0.3.1                    pypi_0    pypi
wcwidth                   0.2.8              pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.6.4              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.9              pyhd8ed1ab_0    conda-forge
wrapt                     1.15.0          py311h2725bcf_1    conda-forge
xarray                    2023.9.0           pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.4                h6314983_3    conda-forge
xlrd                      2.0.1              pyhd8ed1ab_3    conda-forge
xorg-libxau               1.0.11               h0dc2134_0    conda-forge
xorg-libxdmcp             1.1.3                h35c211d_0    conda-forge
xyzservices               2023.10.0          pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h775f41a_0    conda-forge
yaml                      0.2.5                h0d85af4_2    conda-forge
yte                       1.5.1              pyha770c72_2    conda-forge
zeromq                    4.3.5                h93d8f39_0    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h8a1eda9_5    conda-forge
zlib-ng                   2.0.7                hb7f2c08_0    conda-forge
zstd                      1.5.5                h829000d_0    conda-forge
degleris1 commented 11 months ago 
conda list dask
# packages in environment at /Users/degleris/.julia/conda/3/x86_64/envs/pypsa-usa2:
# Name                    Version                   Build  Channel
dask                      2023.7.0           pyhd8ed1ab_0    conda-forge
dask-core                 2023.7.0           pyhd8ed1ab_0    conda-forge
xarray                    2023.6.0           pyhd8ed1ab_0    conda-forge

Downgrading dask and xarray to the above versions seemed to resolve the issue.