Downloading the git
repository:
Download the main repository to somewhere (e.g. into a projects
folder):
(projects) > git clone https://github.com/euronion/trace.git
This repository contains the main workflow, the energy supply chain definitions and all the necessary scripts tying it together.
Setup your python
environment:
a. Install the python
environment manager conda
.
You need a conda
compatible package manager installed.
Recommended is mamba.
Alternatively use miniconda or Anaconda.
b. Setup your python
environment:
conda env create -f envs/environment.yaml
# or if you have mamba installed
mamba env create -f envs/environment.yaml
This installs all the packages and python
dependencies.
Setup your julia
environment (we used julia-1.4
)
a. Download and install julia
following the installation instruction on the Julia website.
b. Open a command shell and navigate into the trace
folder
cd trace
c. Start julia
(trace): julia
d. Open the package manager of julia
julia> ]
e. Register and instantiate the environment
(@v1.4) pkg> activate ./envs
(trace) pkg> instantiate
(Optionatl) Setup a Gurobi solver license. For details and how to obtain a free academic license (if you are eligible), see the documentation. If you don't want to use Gurobi you need to setup an alternative linear solver compatible with PyPSA and change the source code to be comptaible with the solver.
The repository contains the follwing folder and files:
(trace): tree -L 2
.
├── actions # Scripts for building and solving the model
│ ├── attach_supply.py.ipynb
│ ├── backup_run.py
│ ├── combine_GEGIS_demand.py
│ └── ...
├── config # Input configuration for model and scenarios
│ ├── config.cern_link.yaml
│ ├── config.default.yaml
│ └── config.initial_paper.yaml
├── data # Input data for the model
│ ├── technology-data
│ ├── distances.csv
│ ├── efficiencies.csv
│ ├── import_profiles
│ ├── overwrite
│ ├── shipping.csv
│ └── wacc.csv
├── envs # Programming environment definitions for installing/reproducing the model
│ ├── environment.yaml
│ ├── environment.yaml.used
│ ├── JuliaManifest.toml
│ └── JuliaProject.toml
├── escs # ESC definitions which serve as input; will be modified to create ESC-exp-imp specific models
│ ├── hvdc-to-elec
│ ├── hvdc-to-h2
│ ├── pipeline-ch4
│ └── ...
├── LICENSES # License files for all licenses used in this repository
│ ├── CC0-1.0.txt
│ ├── CC-BY-4.0.txt
│ └── GPL-3.0-or-later.txt
├── logs # (created automatically) log files for each model step, scenario and ESC
│ └── ...
├── README.md # You are reading this
├── resources # (created automatically) intermediary files created during model creation
│ └── ...
├── results # (created automatically) Result files for each scenario/year/ESC/exporter-importer;
| | # Contains the optimised PyPSA networks and a CSV file with prominent results
│ ├── default
│ ├── lowhomogeneous
│ ├── results.csv
│ └── ...
├── rules # Snakemake rule directory containing the workflow definitino for the model
│ ├── esc_construction.smk
│ ├── gegis.smk
│ ├── results.smk
│ └── solving.smk
└── Snakefile # Snakemake file to combine all workflow files into the complete workflow
This model uses scenarios to define a number of parameters identical across multiple model runs.
Scenarios are defined in config/config.default.yaml
and config/config.initial_paper.yaml
.
Identically named scenarios from the second file take precedence over scenarios from the first file.
Two central scenarios are the scenario default
and lowhomogeneous
.
Both are similar except for default
assuming 10% WACC (wacc: "homogeneous"
) and
lowhomogeneous
assuming 5% WACC wacc: "lowhomogeneous"
).
The values for wacc: <value>
are references to column entries in data/wacc.csv
.
ESCs are defined in the esc
subfolder as PyPSA networks.
For each ESC, its specific subfolder is parsed and converted into a region-specific PyPSA network representing the ESC.
Following files corresponding to PyPSA components are used for described purposes:
technology-data
repository (CAPEX, FOM, lifetime)config.yaml
actions/create_network.py.ipynb
)actions/create_network.py.ipynb
). The following adjustments are made:
data/distances.csv
and link capital costs per distance are scaled with the distance. - Shipping is special as it is a non-continuous transportation method
- Shipping connections are defined with the non-standard PyPSA `ships.csv` in the respecitve ESC
- `ships.csv` is read by `create_network.py.ipynb` and then translated into comparable PyPSA compoenents
- Internally a ship is represented by a link for loading, a link for unloading and a link for representing energy loss during the journey and a store (cargo hold).
The energy loss during the journey is either the energy required for propulsion or boil-off, whatever is larger.
The idea is that boil-off may be used for propulsion as well and propulsion energy is always provided by the ship's cargo.
- A delivery schedule is exogeneously provided, letting the loading and unloading links work with fixed delay representing the loading/unloading process and
the travel time inbetween.
- Instead of a ship with a fixed maximum capacity we think of it more as a group of ships = convoy which all travel together without an upper limit on capacity.
- Additionally a constraint added in `solve_network.py.ipynb` for the solver ensures that for each convoy the annual loaded and unloaded amounts are equal.
- For each shipping connection, multiple convoys are added such that all possible combinations are accounted for.
In these combinations, the convoys can not load or unload while another convoy is loading/unloading.
- The convoys are scheduled such that a as-constant-as-possible supply may be build by the model, i.e. a convoy arrives always shortly after the previous
convoy left.
- Travel time may be artificially increased to achieve a more constant supply by ship and avoid longer periods were not shipment arrives
at the end of the year. The increase should be negiglible and in the range of a few hours per trip and is also logged for each scenario.
- Additionally convoys may arrive a few hours later to fill gaps in the scheduling (again to smoothen the supply)
- The model is provided with the investment costs per amount of cargo deliverd and may freely choose the amount to transport during for each shippment in each shipping "lane"
- Limitation of the exogeneous scheduling:
At the beginning of each scenario year there might be a larger supply gap of a few hours/days, where no ship arrives due to unfavourable combinations of
journey time and loading/unloading duration
For LOHC transport the chemical costs also have to be taken into account.
This is hard-coded into the model in actions/create_network.py.ipynb
in the method override_costs_for_special_cases(n)
.
The following assumptions are made:
The hard-coding modifies the following costs:
Exporting and importing regions are defined in config/config.default.yaml
under the key
regions:
TRACES:
"AU": "Australia"
"AR": "Argentina"
"DE": "Germany"
"DK": "Denmark"
"EG": "Egypt"
"ES": "Spain"
"MA": "Morocco"
"SA": "Saudi Arabia"
The model is setup to support imports from any of these regions to Germany (DE).
For each region a complete row entry in data/wacc.csv
is necessary.
Further for each export-import relationship which is desired to be investigated, the different types of
distances (as-the-crow-flies, as-the-hake-swims, sea route) are provided via data/distances.csv
.
If a new region is supposed to be supported as importer or exporter, the new region has to be
included in all three locations with a complete entry or else the model will fail with an KeyError
when trying to run the model on a new region with incomplete entries.
Note: This repository relies on
snakemake
as a workflow manager. For more information and usage ofsnakemake
see the documentation.
The model allows to simulate an energy supply chain (ESC) under various conditions.
Data technology costs (CAPEX, FOM, lifetime) are taken from the technology-data GitHub repository
and downloaded automatically into data/technology-data/outputs/costs_<year>.csv
.
Depending on the year
choosen for a scenario run (see below), the corresponding costs_<year>.csv
file is used.
Efficiencies for conversion processes, e.g. synthesis or hydrogen electrolysis are provided via data/efficiencies.csv
.
A single model run or scenario run is a specific choice of the following:
scenario
(from the config.default.yaml
and config.initial_paper.yaml
files)esc
exporter
importer
year
Use the following command where values <...>
are substituted with the specific values:
snakemake -jall results/<scenario>/<year>/<esc>/<exporter>-<importer>/results.csv
# example
snakemake -jall results/default/2030/shipping-lch4/AR-DE/results.csv
The command executes the necessary steps, solves the network(s) for all combinations of ESCs, exporting and importing countries,
extracts the results and merges the main results into a single results/.../results.csv
file for manual or automatic result analysis.
In the same folder as the results.csv
file there will also be saved a network.nc
file.
This file is the solved, i.e. optimised, PyPSA network for the specific case modelled and can be opened using PyPSA
import pypsa
network = pypsa.Network(<path to network.nc file>)
All entries in results.csv
are based on the network.nc
and extracted for convinience using the Python notebook actions/extract_results.py.ipynb
.
Using the snakemake
functionality of parameter spaces one or scenarios runs can be defined and solved easily.
Each case to model and solve is specified by a row entry in scenarios/default.csv
.
Every row must contain a valid entry for all 5 columns, else the workflow will fail.
Then all the scenario runs can be modelled and solved by calling
snakemake -jall results/results.csv
This will create all the individual results/.../results.csv
and results/.../network.nc
files for each scenario run.
In addition it will create a file results/results.csv
containing all individual results files combined into a single file for convinience.
Sensitivity analysis is implemented through different scenarios, see config/config.initial_paper.yaml
example
sensitivity scenarios and the default
scenario in config/config.default.yaml
for all implemented modifiers
in the model.
Scenario runs for the sensitivity analysis are run as standard scenario cases, see the instructions above.
Workflow stops with "Address is already in use" error messages:
In some cases scripts in the snakemake workflow can fail if too many snakemake
rules are started simultaneously.
This issue is related to the use of Jupyter notebook by the model.
This problem cannot easily be avoided. Since the problem is not with the model but with the execution, an effective workaround
is to tell snakemake
to simply try to rerun the failed affected rules by adding the --restart-times <N>
flag added to call, e.g.
snakemake -jall --restart-times 3 results/results.csv
"Python kernel died" during solve_network
:
Memory consumption during solving step can exceeds the memory snakemake
has allocated to the solving process.
The model is setup to increase the reserved memory per repetition of the same solve_network
step if the previous
iteration has failed. Apply the same solution flags to snakemake
as for the "Adress is already in use" problem bove.
<esc>/loads.csv
in MWh/h, conversion table for hydrogen:energy carrier | hourly amount [t] | annual amount [t] | hourly amount [MWh] | annual amount [TWh] | HV assumed |
---|---|---|---|---|---|
H2 | 410.96 | 3.6e6 | 13 698.65 | 120 | LHV |
config/config.default.yaml
actions/attach_supply.py.ipynb
This software is archived in Zenodo. A permanent record of the most recent release version can be found here: .