search

PyPSA / pypsa-usa

PyPSA-USA: An Open-Source Energy System Optimization Model for the United States
https://pypsa-usa.readthedocs.io
MIT License
55 stars 23 forks source link

Sandbox Zenodo Repo Deleted #100

Closed ktehranchi closed 1 year ago

ktehranchi commented 1 year ago

Checklist

The Issue

Somehow the zenodo repository and my account on zenodo are no longer accessible. This changed between Monday and now....

Need to re-upload all the cutout zenodo repositories

https://sandbox.zenodo.org/record/1245777/files/{interconnect}_{cutout}.nc?download=1

Steps To Reproduce

No response

Expected Behavior

No response

Error Message

No response

Anything else?

No response

nomineb commented 1 year ago 
Traceback (most recent call last):
  File "/Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/.snakemake/scripts/tmp8d9wudtc.retrieve_databundles.py", line 40, in <module>
    with zipfile.ZipFile(tarball_fn, "r") as zip_ref:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/zipfile.py", line 1302, in __init__
    self._RealGetContents()
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/zipfile.py", line 1369, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
[Fri Oct 13 15:18:17 2023]
Error in rule retrieve_zenodo_databundles:
    jobid: 8
    output: data/breakthrough_network/base_grid/bus.csv, data/breakthrough_network/base_grid/sub.csv, data/breakthrough_network/base_grid/bus2sub.csv, data/breakthrough_network/base_grid/branch.csv, data/breakthrough_network/base_grid/dcline.csv, data/breakthrough_network/base_grid/demand.csv, data/breakthrough_network/base_grid/plant.csv, data/breakthrough_network/base_grid/solar.csv, data/breakthrough_network/base_grid/wind.csv, data/breakthrough_network/base_grid/hydro.csv, data/breakthrough_network/base_grid/zone.csv, data/gebco/gebco_2023_tid_USA.nc, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_USA_EPSG-4326.tif, data/natura.tiff
    log: logs/retrieve/retrieve_databundles.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/Snakefile, line 107:
Command 'set -euo pipefail;  /Users/nomio/miniforge3/envs/pypsa-usa/bin/python3.11 /Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/.snakemake/scripts/tmp8d9wudtc.retrieve_databundles.py' returned non-zero exit status 1.
  File "/Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/Snakefile", line 107, in __rule_retrieve_zenodo_databundles
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/concurrent/futures/thread.py", line 58, in run
Removing output files of failed job retrieve_zenodo_databundles since they might be corrupted:
data/natura.tiff
100% (100 of 100) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
  6% (6 of 100) |#                       | Elapsed Time: 0:00:00 ETA:   0:00:10[Fri Oct 13 15:18:17 2023]
Finished job 18.
1 of 22 steps (5%) done
 25% (25 of 100) |#####                  | Elapsed Time: 0:00:00 ETA:   0:00:02EIA930_BALANCE_2023_Jan_Jun.csv downloaded successfully!
 98% (98 of 100) |###################### | Elapsed Time: 0:00:01 ETA:   0:00:00EIA930_BALANCE_2022_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2022_Jan_Jun.csv downloaded successfully!
100% (100 of 100) |######################| Elapsed Time: 0:00:04 Time:  0:00:04
[Fri Oct 13 15:18:21 2023]
Finished job 17.
2 of 22 steps (9%) done
EIA930_BALANCE_2021_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2021_Jan_Jun.csv downloaded successfully!
EIA930_BALANCE_2020_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2020_Jan_Jun.csv downloaded successfully!
EIA930_BALANCE_2019_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2019_Jan_Jun.csv downloaded successfully!
EIA930_BALANCE_2018_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2018_Jan_Jun.csv downloaded successfully!
EIA930_BALANCE_2017_Jul_Dec.csv downloaded successfully!
EIA930_BALANCE_2017_Jan_Jun.csv downloaded successfully!
resources/eia/EIA_DMD_2018.csv saved successfully!
resources/eia/EIA_DMD_2023.csv saved successfully!
resources/eia/EIA_DMD_2021.csv saved successfully!
resources/eia/EIA_DMD_2019.csv saved successfully!
resources/eia/EIA_DMD_2022.csv saved successfully!
resources/eia/EIA_DMD_2017.csv saved successfully!
resources/eia/EIA_DMD_2020.csv saved successfully!
[Fri Oct 13 15:18:39 2023]
Finished job 20.
3 of 22 steps (14%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-13T151815.776823.snakemake.log
nomineb commented 1 year ago

Hi Kamran! Got this error when running the workflow after our call. 

Traceback (most recent call last):
  File "/Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/.snakemake/scripts/tmpcj91d6s6.build_renewable_profiles.py", line 300, in <module>
    profile, capacities = func(
                          ^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/atlite/convert.py", line 602, in pv
    return cutout.convert_and_aggregate(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/atlite/convert.py", line 174, in convert_and_aggregate
    results = aggregate_matrix(da, matrix=matrix, index=index)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/atlite/aggregate.py", line 19, in aggregate_matrix
    return xr.apply_ufunc(
           ^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/xarray/core/computation.py", line 1249, in apply_ufunc
    return apply_dataarray_vfunc(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/xarray/core/computation.py", line 308, in apply_dataarray_vfunc
    result_var = func(*data_vars)
                 ^^^^^^^^^^^^^^^^
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/site-packages/xarray/core/computation.py", line 754, in apply_variable_ufunc
    raise ValueError(
ValueError: dimension spatial on 0th function argument to apply_ufunc with dask='parallelized' consists of multiple chunks, but is also a core dimension. To fix, either rechunk into a single array chunk along this dimension, i.e., ``.chunk(dict(spatial=-1))``, or pass ``allow_rechunk=True`` in ``dask_gufunc_kwargs`` but beware that this may significantly increase memory usage.
[Fri Oct 13 18:57:30 2023]
Error in rule build_renewable_profiles:
    jobid: 14
    input: resources/western/elec_base_network.nc, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_USA_EPSG-4326.tif, data/natura.tiff, resources/western/country_shapes.geojson, resources/western/offshore_shapes.geojson, resources/western/regions_onshore.geojson, cutouts/western_era5_2019.nc
    output: resources/western/profile_solar.nc
    log: logs/western/build_renewable_profile_solar.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/Snakefile, line 409:
Command 'set -euo pipefail;  /Users/nomio/miniforge3/envs/pypsa-usa/bin/python3.11 /Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/.snakemake/scripts/tmpcj91d6s6.build_renewable_profiles.py' returned non-zero exit status 1.
  File "/Users/nomio/Documents/REU/Fall_2324/pypsa-usa/workflow/Snakefile", line 409, in __rule_build_renewable_profiles
  File "/Users/nomio/miniforge3/envs/pypsa-usa/lib/python3.11/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-13T185103.899139.snakemake.log
nomineb commented 1 year ago

build_renewable_profile_solar.log