Error trying to download file powerplants.csv in rule build_powerplants

[AFU3141]

Checklist

• [x] I am using the current master branch or the latest release. Please indicate.
• [x] I am running on an up-to-date pypsa-eur environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

I have installed the recent version v0.11.0 of pypsa eur from github and have installed the conda environment by using the corresponding yaml file.

I ran the standard electricity only tutorial as in https://pypsa-eur.readthedocs.io/en/latest/tutorial.html. While running snakemake -call results/test-elec/networks/elec_s_6_eclcopt.nc --configfile config/test/config.electricity.yaml the rule build_powerplants was not executed successfully.

It seems that the file https://raw.githubusercontent.com/FRESNA/powerplantmatching/v0.0.0/powerplants.csv cannot be downloaded.

Error Message

localrule build_powerplants:
    input: resources/test/networks/base.nc, data/custom_powerplants.csv
    output: resources/test/powerplants.csv
    log: logs/test/build_powerplants.log
    jobid: 0
    reason: Forced execution
    resources: mem_mb=7000, mem_mib=6676, disk_mb=1000, disk_mib=954, tmpdir=/tmp, slurm_partition=C7

INFO:pypsa.io:Imported network base.nc has buses, carriers, lines, shapes, transformers
INFO:powerplantmatching.collection:Retrieving data from https://raw.githubusercontent.com/FRESNA/powerplantmatching/v0.0.0/powerplants.csv
ERROR:root:Uncaught exception
Traceback (most recent call last):
  File " /.snakemake/scripts/tmpbjvy9qfq.build_powerplants.py", line 181, in <module>
    pm.powerplants(from_url=True)
  File " /lib/python3.12/site-packages/powerplantmatching/collection.py", line 206, in powerplants
    pd.read_csv(url, index_col=0)
  File " /lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 620, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1620, in __init__
    self._engine = self._make_engine(f, self.engine)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1880, in _make_engine
    self.handles = get_handle(
                   ^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/common.py", line 728, in get_handle
    ioargs = _get_filepath_or_buffer(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/common.py", line 384, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
         ^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/site-packages/pandas/io/common.py", line 289, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 521, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 630, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 559, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File " /lib/python3.12/urllib/request.py", line 639, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
RuleException:
CalledProcessError in file  /rules/build_electricity.smk, line 65:
Command 'set -euo pipefail;   /bin/python3.12  /.snakemake/scripts/tmpbjvy9qfq.build_powerplants.py' returned non-zero exit status 1.

Error in rule build_powerplants:
    jobid: 0
    input: resources/test/networks/base.nc, data/custom_powerplants.csv
    output: resources/test/powerplants.csv
    log: logs/test/build_powerplants.log (check log file(s) for error details)
    conda-env:  /.snakemake/conda/610cc29d38c3a2f88ad2dec9b15e300f_

[fneum]

In the config file, costs: version: should be set to a valid version tag or commit-hash of the powerplantmatching tutorial. The current default is v0.9.0 in config.default.yaml.

[fneum]

reopen if issue persists

[AFU3141]

Putting costs: v0.9.0 into the config file does not solve the download issue. Even other versions or master does not work. Apparently, the costs version parameter is not used in the target URL https://raw.githubusercontent.com/FRESNA/powerplantmatching/v0.0.0/powerplants.csv. So the downlaod is started for version v0.0.0.

INFO:pypsa.io:Imported network base.nc has buses, carriers, lines, links, shapes, transformers INFO:powerplantmatching.collection:Retrieving data from https://raw.githubusercontent.com/FRESNA/powerplantmatching/v0.0.0/powerplants.csv ERROR:root:Uncaught exception Traceback (most recent call last): File "/trinity/home/fuhrand/pypsa-eur-v0.11.0/.snakemake/scripts/tmpuuw57e0q.build_powerplants.py", line 181, in pm.powerplants(from_url=True) File "/trinity/home/fuhrand/micromamba/envs/pypsa-eur-v0.11.0/lib/python3.12/site-packages/powerplantmatching/collection.py", line 206, in powerplants pd.read_csv(url, index_col=0) File "/trinity/home/fuhrand/micromamba/envs/pypsa-eur-v0.11.0/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv return _read(filepath_or_buffer, kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[AFU3141]

Please reopen

[shithikanta]

Yes, I also face the same issue, even after using the latest version of the repo. attaching the error details for your reference. error details.txt Please reopen it.