pypsa-meets-earth / pypsa-earth

PyPSA-Earth: A flexible Python-based open optimisation model to study energy system futures around the world.
https://pypsa-earth.readthedocs.io/en/latest/
224 stars 175 forks source link

Snakemake Warning #1114

Open GbotemiB opened 1 day ago

GbotemiB commented 1 day ago

Checklist

Describe the Bug

I notice the warning after running snakemake -j8 solve_sector_networks for sector-couple.

Please provide a description of what the bug is and add a minimal example/command for reproducing the bug.

Error Message

If applicable, paste any terminal output to help illustrating your problem. In some cases it may also be useful to share your list of installed packages: conda list.

  /Users/gbotemi/miniconda3/envs/pypsa-earth-0.4.1/lib/python3.10/site-packages/pulp/apis/gurobi_api.py:170: UserWarning: GUROBI error: Unknown parameter '_test'.
    warnings.warn("GUROBI error: {}.".format(e))
  /Users/gbotemi/miniconda3/envs/pypsa-earth-0.4.1/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
    warnings.warn(
  Demand data folder: data/ssp2-2.6/2030/era5_2013, load path is ['data/ssp2-2.6/2030/era5_2013/Africa.nc'].
  Expected files: Africa.nc
  Building DAG of jobs...
  File path results//postnetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//configs/config.yaml contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//postnetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/solve_network/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_solver.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_python.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_memory.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_presec.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/prepare_network/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//prenetworks/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_presec.nc contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//configs/scenario.done contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//configs/config.yaml contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//configs contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//configs/config.yaml contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/copy_config contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/solve_network/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_solver.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_python.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//logs/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB_10export_memory.log contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/prepare_network/elec_s_10_ec_lcopt_Co2L-3H_144H_2030_0.071_AB contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
  File path results//benchmarks/copy_config contains double '/'. This is likely unintended. It can also lead to inconsistent results of the file-matching approach used by Snakemake.
ekatef commented 1 day ago

Hey @GbotemiB, thanks for opening the issue! I have also experienced this warning. Are you willing to investigate that?

GbotemiB commented 1 day ago

That is fine by me. I will look into it.

GbotemiB commented 19 hours ago

Hi @ekatef,

I found the solution, which is to remove the / before the RDIR

SDIR = config["summary_dir"].strip("/") + f"/{RDIR}/"
RESDIR = config["results_dir"].strip("/") + f"/{RDIR}/"

It will look like this

SDIR = config["summary_dir"].strip("/") + f"{RDIR}/"
RESDIR = config["results_dir"].strip("/") + f"{RDIR}/"
ekatef commented 10 hours ago

Hi @ekatef,

I found the solution, which is to remove the / before the RDIR

SDIR = config["summary_dir"].strip("/") + f"/{RDIR}/"
RESDIR = config["results_dir"].strip("/") + f"/{RDIR}/"

It will look like this

SDIR = config["summary_dir"].strip("/") + f"{RDIR}/"
RESDIR = config["results_dir"].strip("/") + f"{RDIR}/"

Hey @GbotemiB, perfect, thanks for spotting! Feel free to open a PR with a fix 😄