search

pypsa-meets-earth / pypsa-earth

PyPSA-Earth: A flexible Python-based open optimisation model to study energy system futures around the world.
https://pypsa-earth.readthedocs.io/en/latest/
207 stars 167 forks source link

CSP integration into PyPSA-Earth Workflow #997

Closed GbotemiB closed 1 month ago

GbotemiB commented 3 months ago

Closes #145 (if applicable).

Changes proposed in this Pull Request

This PR aims to integrate the CSP technology into the PyPSA-Earth Workflow

Checklist

GbotemiB commented 3 months ago

By CSP time series, do you mean if a renewable profile network file is being created for CSP?

If so, yes. At the moment, I can confirm that a CSP renewable profile network is created alongside other renewable profiles.

davide-f commented 3 months ago

By CSP time series, do you mean if a renewable profile network file is being created for CSP?

If so, yes. At the moment, I can confirm that a CSP renewable profile network is created alongside other renewable profiles.

Cool great work 👯‍♂️

ekatef commented 3 months ago

Hello @GbotemiB! Great progress 😄

A couple of technical comments: 1) It looks like costs.csv has been modified more deeply as it has been probably intended. Could you please fix that? 2) Regarding the costs inputs, this part of technology-data repo seems to be relevant. But there a need to fetch data for 2030 to match with PyPSA-Earth conventions for data/costs.csv.

GbotemiB commented 3 months ago

Hi @ekatef, regarding the data in this part which you mentioned. Is there an already existing method used to fetch this data into the PyPSA-Earth Workflow?

ekatef commented 3 months ago

@GbotemiB perfect, thanks a lot for the fix for costs.csv.

Have checked add_extra_components and it looks like are are absolutely right about the type of n.mapp() returns: that is in fact an index. Not sure that is the safest approach, but looks quite elegant.

Great that you have found a workaround for the buses specification. Probably, we can improve it a bit, and that is a moment where out drafting work may get useful. Currently, we have electrical buses only, while there is a need to have also heat buses. Which means the need to add a set of buses like it's done in add_extra_components: https://github.com/pypsa-meets-earth/pypsa-earth/blob/d3d718b8e22fab344991a73a9f9637bf6f22ecb4/scripts/add_extra_components.py#L108-L112

Regarding the underwater fraction, that is a parameter which needs to be defined for submarine HVDC lines. Link component may be used to represent HVDC lines in the network. There are also links which connect the stores, but those are added after simplify_network. To me, it looks like we need to implement a method to exclude links corresponding to CSP from treatment in this part: https://github.com/pypsa-meets-earth/pypsa-earth/blob/d3d718b8e22fab344991a73a9f9637bf6f22ecb4/scripts/simplify_network.py#L439-L441 As a suggestion, filtering by carrier may be a good approach. What do you think?

GbotemiB commented 3 months ago

Thanks @ekatef for the review.

Question regarding the electrical buses, are the buses in the renewable profile network an electrical bus? I thought the buses from the renewable_profile will be as a result of the profile type, in this case heat for the csp profile.

ekatef commented 3 months ago

Thanks @ekatef for the review.

Question regarding the electrical buses, are the buses in the renewable profile network an electrical bus? I thought the buses from the renewable_profile will be as a result of the profile type, in this case heat for the csp profile.

Buses used in build_renewable_profiles are bus_regions: https://github.com/pypsa-meets-earth/pypsa-earth/blob/d3d718b8e22fab344991a73a9f9637bf6f22ecb4/Snakefile#L457-L461

These regions are built in build_bus_regions as Voronoi polygons around nodes of power grid (or as administrative regions if alternative clustering is considered): https://github.com/pypsa-meets-earth/pypsa-earth/blob/d3d718b8e22fab344991a73a9f9637bf6f22ecb4/scripts/build_bus_regions.py#L198-L207

So, buses are a mapping of the power grid nodes into space. As an example of the bus polygons overlaid by the network nodes:

image

Each polygon aka bus area is used to collect the available renewable potential to simulate a renewable generator which can be commissioned at a corresponding grid node.

Does it clarify? Buses are one of the major concepts we are using for modeling, but not sure our current explanations are clear enough. So, feel free absolutely to ask further questions if needed.

GbotemiB commented 3 months ago

Thanks @ekatef. Much clearer.

GbotemiB commented 3 months ago

Hello @ekatef, I have added the heat buses. I defined the carrier for the heat bus as csp.

GbotemiB commented 3 months ago

In the simplify_network.py, it seems inserting the links before simplify_electricity triggers other issues.

Traceback (most recent call last):
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 890, in <module>
    n, simplify_links_map = simplify_links(
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 413, in simplify_links
    for b, buses, dc_edges in split_links(labels.index[labels == lbl]):
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 380, in split_links
    if m not in nodes or m in seen or contains_ac(ls):
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 321, in contains_ac
    return any(list(map(lambda x: is_ac_branch(x), ls)))
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 321, in <lambda>
    return any(list(map(lambda x: is_ac_branch(x), ls)))
  File "/Users/gbotemi/Documents/code/PYPSA/CSP/pypsa-earth/.snakemake/scripts/tmpybgzrdcz.simplify_network.py", line 315, in is_ac_branch
    return n.links.loc[x[1]].dc != True
  File "/Users/gbotemi/miniconda3/envs/pypsa-earth-osm/lib/python3.10/site-packages/pandas/core/generic.py", line 6296, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'dc'
GbotemiB commented 3 months ago

A short summary of discussion on CSP task with @ekatef and @davide-f .

The CSP task would involve adding links, stores, and generators to the csp bus, which would be connected to the network. But the current PyPSA Earth Workflow currently adds links and stores in the add_extra_components script while it adds just a generator in the add_electricity script.

There are currently two approaches to this.

  1. Add everything related to the csp in the add_electricity script. Going through this process will lead to a lot of adjustments of simplify_network like adding dc column to CSP links or filtering links by carrier. This will lead to a lot of changes in simplify_network script because it will disrupt the workflow.
  2. Add the generators for the csp as electrical generators in the add_electricity script, then in add_extra_components script, switch the electrical generators to csp generators and also add the links and stores for csp.

To implement point 2, we can consider implementation in two steps, when first we add a simplified representation to add_electricity, and after improve it, also adding links and stores in add_extra_components.

ekatef commented 3 months ago

Perfect @GbotemiB! Thanks for such a detailed account, and many thanks to @davide-f for the most insightful discussion.

As a little additional comment, we have also discussed that having m option for clustering means a need to have a dedicated bus for every CSP generator.

Citing the docs for {clusters} wildcard: If an m is placed behind the number of clusters (e.g. 100m), generators are only moved to the clustered buses but not aggregated by carrier; i.e. the clustered bus may have more than one e.g. wind generator. The implementation looks like: https://github.com/pypsa-meets-earth/pypsa-earth/blob/d3d718b8e22fab344991a73a9f9637bf6f22ecb4/scripts/cluster_network.py#L693-L694

GbotemiB commented 3 months ago

Hello @davide-f @katia,

Is there a method in pypsa to reassign buses of network components?

I am open to thoughts on how to remove and attach csp buses to the electrical generators.

davide-f commented 3 months ago

Hello @davide-f @Katia,

Is there a method in pypsa to reassign buses of network components?

I am open to thoughts on how to remove and attach csp buses to the electrical generators.

You can do that by using: n.generators.loc[csp_index, "bus"] = new_csp_bus_vector

For example: new_csp_bus_vector = csp_index + " csp"

GbotemiB commented 3 months ago

Hello @ekatef @davide-f,

Everything seems to work now. I also added option for advanced or simple in the config for csp. I think the PR is ready for review.

GbotemiB commented 3 months ago

I think the CI is failing because the cost file is missing csp rows

ekatef commented 3 months ago

Perfect @GbotemiB! Congratulations with making the approach work 🎉

My feeling is that the design is basically complete and we are very close to finalise it's implementation. Have added some technical comments which relate to polishing of the current approach.

ekatef commented 3 months ago

The next step is data checking and creation of an example (which can also serve as a testing approach).

In part of data: 1) we need to ensure that the Copernicus grid codes are appropriate; 2) would be great to cross-check values of capacity_per_sqkm and correction_factor against the published data, keeping in sight #362 and #361 may be useful for that [as a side note, I do not completely understand the meaning of correction_factor; could be a good idea to discuss it and improve documentation on that].

Regarding the example, we can probably add a section on CSP into build_renewable_profiles, if CSP is viable for Nigeria (not sure about that), or create a dedicated example considering an area with higher direct radiation values.

As an advanced comment, we are considering only one type of CSP technologies, namely CSP tower. I wonder if it would it be possible to add also parabolic through. What do you think?

GbotemiB commented 3 months ago

Thanks @ekatef for the support.

Regarding adding another csp technology, I can look into it. It shouldn't be too difficult to handle.

ekatef commented 3 months ago

Thanks @ekatef for the support.

Regarding adding another csp technology, I can look into it. It shouldn't be too difficult to handle.

My pleasure!

Perfect, thanks! It would be definitely great to have both CSP technologies implemented.

ekatef commented 2 months ago

A technical comment: I'm thinking that CSP are especially valuable thanks to the storage which is their part by design. So, CSP generation do not follow all the cloudiness variations immediately, but provides some constant output. That is a nice contrast to dynamics of PV or wind generation and facilitates operation of the power system a lot.

Moreover, CSP can be used as a dispatchable source, which can dramatically facilitate integration of RES into a power system, e.g. as described in this report.

It would be great to ensure that the modelled behaviour of CSP demonstrates the mentioned features in fact. Ideally, CSP generation should more or less follow the demand.

I suspect that the simplified CSP representation doesn't satisfy this condition, as CSP potential is likely to be as intermittent, as one of PV generation, since we don't use any smoothing. Could be probably daily or multi-daily averaging be applied to CSP potential in the simplified representation to emulate smoothing effects of the storage?

@GbotemiB @davide-f what is your feeling about that?

ekatef commented 2 months ago

An additional academic output which can probably help for quantitative analysis of our results is this Copernicus preprint. It is focused on projections for the CSP potential, but there are also some suggestions for land usage and some corrections used to evaluate the potential.

GbotemiB commented 1 month ago

@ekatef @davide-f @pz-max

To visualize the renewable potential of CSP, I have uploaded the notebooks here

To visualize the generation mix from the modeling output (CSP inclusive), I have uploaded the notebooks here

ekatef commented 1 month ago

@ekatef @davide-f @pz-max

To visualize the renewable potential of CSP, I have uploaded the notebooks here

To visualize the generation mix from the modeling output (CSP inclusive), I have uploaded the notebooks here

Fantastic work @GbotemiB!! I think, the outputs look pretty good. Some technical points can be probably improved, like adjusting of the plots to account for the energy balance and some parameter fixes. However, I agree with @davide-f that we should not to go into too much details with parameters investigation: it should be enough to cross-check capacity_per_sqkm and the land codes.

What may be of concern are these CSP spikes in the negative area of the dispatch plots. It looks like CSP storage serves not only CSP needs, but also to the energy system in general. That is probably good from the system perspective, but not exactly realistic.

davide-f commented 1 month ago

@ekatef @davide-f @pz-max To visualize the renewable potential of CSP, I have uploaded the notebooks here To visualize the generation mix from the modeling output (CSP inclusive), I have uploaded the notebooks here

Fantastic work @GbotemiB!! I think, the outputs look pretty good. Some technical points can be probably improved, like adjusting of the plots to account for the energy balance and some parameter fixes. However, I agree with @davide-f that we should not to go into too much details with parameters investigation: it should be enough to cross-check capacity_per_sqkm and the land codes.

What may be of concern are these CSP spikes in the negative area of the dispatch plots. It looks like CSP storage serves not only CSP needs, but also to the energy system in general. That is probably good from the system perspective, but not exactly realistic.

Great @GbotemiB :D Regarding the stores, in the plot, it is recommended to have the plot of the link rather than the store. For internal check, on the other hand, we can have a plot where only the csp generator, the csp store and the csp link are plotted to guarantee its operation. I think it makes sense the store to exchange sign, but the link must not.

GbotemiB commented 1 month ago

Considering Algeria, Here is the result for the links.

image

image image The timeseries for the links actually cancel out.

This is the plot for the generator image

This is the plot for the stores image

This is the generation mix for Algeria. Initially, I used stores for csp, when it should have been generator just like it was done for solar pv. image

ekatef commented 1 month ago

Fantastic @GbotemiB! Great work and agree with @davide-f that we are very close to finalise 😄

Thanks a lot to you both for cross-checking operation of CSP unit. It looks like the model prefers to size CSP in a way which prevents the plant from running continuously. Not sure it's exactly realistic from the engineering perspective, but that is a rather advanced question which is clearly out of scope for the PR.

Currently going through the final review round mainly to ensure to ensure integrity of the implementation improvements.

ekatef commented 1 month ago

Hello @GbotemiB!

Looks perfect 😄 Thanks for the final touches.

There is a small question left on the test: have you been thinking to include CSP in CI, but wanted to keep it light, correct?

The only change I'm thinking about is adjusting capacity_per_sqkm to ensure that we capture the differences with PV. A citation from [Franke et al 2024]: "For CSP power plants, the highest capacity is 6.4 MW/km2 as this technology needs more area per plant, while for PVs they report 12.26 MW/km2". I'd probably take CSP/PV ratio from there and adjust our capacity_per_sqkm accordingly -> multiplying the current values for CSP on 6.4/12.26 = 0.52. What do you think?

ekatef commented 1 month ago

Two more advanced modeling points I'd track as features requests:

  1. Landuse availability for CSP should include the slope, which can be implemented using gebco dataset.
  2. Some additionally constraints should be applied for CSP model to make sure that its operation is more realistic from thermal engineering perspective.

But that is another story 🙂

GbotemiB commented 1 month ago

Hello @GbotemiB!

Looks perfect 😄 Thanks for the final touches.

There is a small question left on the test: have you been thinking to include CSP in CI, but wanted to keep it light, correct?

The only change I'm thinking about is adjusting capacity_per_sqkm to ensure that we capture the differences with PV. A citation from [Franke et al 2024]: "For CSP power plants, the highest capacity is 6.4 MW/km2 as this technology needs more area per plant, while for PVs they report 12.26 MW/km2". I'd probably take CSP/PV ratio from there and adjust our capacity_per_sqkm accordingly -> multiplying the current values for CSP on 6.4/12.26 = 0.52. What do you think?

About including CSP into the CI, I thought about it. I noticed that the rest of the renewable technology are already in the CI. All it took was adding csp to landlock config.

With regards to the capacity_per_sqkm for csp. I agree that we can reduce the capacity_per_sqkm with a factor of 0.52, which results to 2.392.

davide-f commented 1 month ago

To me this PR seems fine, to align @ekatef feel free to merge when appropriate :)

ekatef commented 1 month ago

Merged! 😄 Thanks a lot for an amazing contribution @GbotemiB 🎉 🎉 🎉 Thank you @davide-f and @pz-max for the outstanding support 🙂

pz-max commented 4 weeks ago

Hi guys, I think we need a more generic version of CSP. Can someone raise an issue?

Current implementation is quite static but of course a good start.

davide-f commented 4 weeks ago

Good point @pz-max . Issue added here: https://github.com/pypsa-meets-earth/pypsa-earth/issues/1038

Tough I'm unsure about the urgency, indeed the current implementation is a good start. do we have some users on that?