Zenodo repository - not the right file

[nribei]

Hello, Thanks a lot for sharing your model :)

As a first step, I am trying to make your model runs but I have some issues with the data downloaded from the Copernicus Climate Change Service and the Climate Data Store.

Many Thanks, Nathan

[matteodefelice]

Dear Nathan, I would suggest creating an account to the C3S CDS to download the updated data: Climate and energy indicators for Europe from 1979 to present derived from reanalysis (copernicus.eu) https://cds.climate.copernicus.eu/cdsapp#!/dataset/sis-energy-derived-reanalysis?tab=overview

https://cds.climate.copernicus.eu/cdsapp#!/dataset/sis-energy-derived-reanalysis?tab=overview It's very easy and free: How to use the CDS API | Copernicus Climate Data Store https://cds.climate.copernicus.eu/api-how-to

On Thu, Nov 23, 2023 at 11:25 AM nribei @.***> wrote:

Hello, Thanks a lot for sharing your model :)

As a first step, I am trying to make your model runs but I have some issues with the data downloaded from the Copernicus Climate Change Service and the Climate Data Store.

thats not the same name files in the code, and on the zenodo repository. Its just an issue of name ? Or I can not use them ?

Many Thanks, Nathan

— Reply to this email directly, view it on GitHub https://github.com/matteodefelice/pypsa-entsoe/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQQYAS4OPAYINYAZBFVF6LYF4QAJAVCNFSM6AAAAAA7XRAFB2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAYDOOJQGE4TSNA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Matteo

[nribei]

It worked ! Thanks.

Does your script run even today ?

I have this mistake: Traceback (most recent call last): File "C:\Users\nribei\PycharmProjects\pythonProject1\stacktesty.py", line 152, in cap_this = metenergy_data.get_capacity_entsoe(zone = COUNTRY, year = int(YEAR), MY_API_KEY=ENTSOE_API_KEY) File "C:\Users\nribei\PycharmProjects\pythonProject1\metenergy_data.py", line 312, in get_capacity_entsoe cap = client.query_installed_generation_capacity(zone, File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\decorators.py", line 100, in year_wrapper frame = func(*args, start=_start, end=_end, **kwargs) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\entsoe.py", line 1381, in query_installed_generation_capacity df = parse_generation(text) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 138, in parse_generation ts = _parse_generation_timeseries(soup, per_plant=per_plant, include_eic=include_eic) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 751, in _parse_generation_timeseries series.index = _parse_datetimeindex(soup) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 879, in _parse_datetimeindex elif pd.Timedelta(delta) >= pd.Timedelta('1D') and end.hour == start.hour + 1: File "pandas_libs\tslibs\timedeltas.pyx", line 1766, in pandas._libs.tslibs.timedeltas.Timedelta.new File "pandas_libs\tslibs\timedeltas.pyx", line 646, in pandas._libs.tslibs.timedeltas.parse_timedelta_string File "pandas_libs\tslibs\timedeltas.pyx", line 694, in pandas._libs.tslibs.timedeltas.timedelta_from_spec ValueError: Units 'M', 'Y' and 'y' do not represent unambiguous timedelta values and are not supported.

Process finished with exit code 1

[matteodefelice]

I will check and let you know, unfortunately those things get broken easily due to API changes and pandas versions...

On Fri, 24 Nov 2023, 11:34 nribei, @.***> wrote:

It worked ! Thanks.

Does your script run even today ?

I have this mistake: Traceback (most recent call last): File "C:\Users\nribei\PycharmProjects\pythonProject1\stacktesty.py", line 152, in cap_this = metenergy_data.get_capacity_entsoe(zone = COUNTRY, year = int(YEAR), MY_API_KEY=ENTSOE_API_KEY) File "C:\Users\nribei\PycharmProjects\pythonProject1\metenergy_data.py", line 312, in get_capacity_entsoe cap = client.query_installed_generation_capacity(zone, File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\decorators.py", line 100, in year_wrapper frame = func(*args, start=_start, end=_end, **kwargs) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\entsoe.py", line 1381, in query_installed_generation_capacity df = parse_generation(text) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 138, in parse_generation ts = _parse_generation_timeseries(soup, per_plant=per_plant, include_eic=include_eic) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 751, in _parse_generation_timeseries series.index = _parse_datetimeindex(soup) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 879, in _parse_datetimeindex elif pd.Timedelta(delta) >= pd.Timedelta('1D') and end.hour == start.hour

• 1: File "pandas_libs\tslibs\timedeltas.pyx", line 1766, in pandas._libs.tslibs.timedeltas.Timedelta.new File "pandas_libs\tslibs\timedeltas.pyx", line 646, in pandas._libs.tslibs.timedeltas.parse_timedelta_string File "pandas_libs\tslibs\timedeltas.pyx", line 694, in pandas._libs.tslibs.timedeltas.timedelta_from_spec ValueError: Units 'M', 'Y' and 'y' do not represent unambiguous timedelta values and are not supported.

Process finished with exit code 1

— Reply to this email directly, view it on GitHub https://github.com/matteodefelice/pypsa-entsoe/issues/1#issuecomment-1825465880, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQQYAR4JI3NY2UFERTH47TYGBZ2HAVCNFSM6AAAAAA7XRAFB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRVGQ3DKOBYGA . You are receiving this because you commented.Message ID: @.***>

[matteodefelice]

Just tried, it worked. Python 3.9, Entsoe-py 0.5.10, pandas 2.0.2

On Fri, Nov 24, 2023 at 11:34 AM nribei @.***> wrote:

It worked ! Thanks.

Does your script run even today ?

I have this mistake: Traceback (most recent call last): File "C:\Users\nribei\PycharmProjects\pythonProject1\stacktesty.py", line 152, in cap_this = metenergy_data.get_capacity_entsoe(zone = COUNTRY, year = int(YEAR), MY_API_KEY=ENTSOE_API_KEY) File "C:\Users\nribei\PycharmProjects\pythonProject1\metenergy_data.py", line 312, in get_capacity_entsoe cap = client.query_installed_generation_capacity(zone, File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\decorators.py", line 100, in year_wrapper frame = func(*args, start=_start, end=_end, **kwargs) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\entsoe.py", line 1381, in query_installed_generation_capacity df = parse_generation(text) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 138, in parse_generation ts = _parse_generation_timeseries(soup, per_plant=per_plant, include_eic=include_eic) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 751, in _parse_generation_timeseries series.index = _parse_datetimeindex(soup) File "C:\Users\nribei\PycharmProjects\pythonProject1\venv\lib\site-packages\entsoe\parsers.py", line 879, in _parse_datetimeindex elif pd.Timedelta(delta) >= pd.Timedelta('1D') and end.hour == start.hour

• 1: File "pandas_libs\tslibs\timedeltas.pyx", line 1766, in pandas._libs.tslibs.timedeltas.Timedelta.new File "pandas_libs\tslibs\timedeltas.pyx", line 646, in pandas._libs.tslibs.timedeltas.parse_timedelta_string File "pandas_libs\tslibs\timedeltas.pyx", line 694, in pandas._libs.tslibs.timedeltas.timedelta_from_spec ValueError: Units 'M', 'Y' and 'y' do not represent unambiguous timedelta values and are not supported.

Process finished with exit code 1

— Reply to this email directly, view it on GitHub https://github.com/matteodefelice/pypsa-entsoe/issues/1#issuecomment-1825465880, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQQYAR4JI3NY2UFERTH47TYGBZ2HAVCNFSM6AAAAAA7XRAFB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRVGQ3DKOBYGA . You are receiving this because you commented.Message ID: @.***>

-- Matteo

[nribei]

Hello ! It works well, thanks. It's very helpful.

I am starting to modelise one country with my data (and cutting all links from entsoe), starting from your code. But I'm having trouble understanding variables and their units.

I have some questions :

• You seem to use the same power plant availability all over the period, right ? (I have each plant with its own availability on an hourly basis)

• Why do you use efficiency and marginal cost ? (for me you dont need efficiency if you gave as an input the marginal cost)

• Regarding ramp_limit_up & down, for a ramp-up of a CCGT for example, I see 0.15. Does that mean that if my CCGT has a p_nom of 400MW, and I turn on the plant, the first hour will be 60MW and then the second hour 400MW?

• Is p_min_pu the Pmin of a power plant ? For example a CCGT of 400MW, has a Pmin of 200MW, so the p_min_pu should be 0.5 ? (in your input I see 0.029, so I am not sure about your unit)

• In the file (ramping-2020_2022.csv), what does represent max_up, max_down , q999_up , q999_down ? From what I understand thats the maximal variation (up&down) of production (in MW) on the aggregation of all capacity per type of generator based on historical ?

• What is "link" in your generator modelisation ?

• Regarding "other_map", do these correspond to generators that need to be added because the capacity given by entsoe is insufficient?

• Last question, could you explain me how you modelise Hydro dam and STEP (pump-turbine station)?

Many Thanks !

[matteodefelice]

Hello, happy to hear that the code is working. I will try to answer your questions!

You seem to use the same power plant availability all over the period, right ? (I have each plant with its own availability on an hourly basis)

Yes, it is almost impossible to infer availability profiles from ENTSO-E data, thus I assume constant availability.

Why do you use efficiency and marginal cost ? (for me you dont need efficiency if you gave as an input the marginal cost)

Well, efficiency and marginal costs define two distinct aspects. In my code, efficiency is not really useful, you can get rid of it without big issues.

Regarding ramp_limit_up & down, for a ramp-up of a CCGT for example, I see 0.15. Does that mean that if my CCGT has a p_nom of 400MW, and I turn on the plant, the first hour will be 60MW and then the second hour 400MW?

The documentation describes ramp_limit_up as Maximum active power increase from one snapshot to the next, per unit of the nominal power.. In other words, a ramp-up of 0.15 means that the generator can increase its capacity of 15% (of the nominal capacity) every time-step. In your example would be 60 MW every hour. The values are inferred by ENTSO-E TP data.

Is p_min_pu the Pmin of a power plant ? For example a CCGT of 400MW, has a Pmin of 200MW, so the p_min_pu should be 0.5 ? (in your input I see 0.029, so I am not sure about your unit)

The documentation describes p_min_pu as The minimum output for each snapshot per unit of p_nom, its dimensionless. In your case, p_min_pu of 0.029 means that that CCGT cannot go below 400 * 0.029 MW.

In the file (ramping-2020_2022.csv), what does represent max_up, max_down , q999_up , q999_down ? From what I understand thats the maximal variation (up&down) of production (in MW) on the aggregation of all capacity per type of generator based on historical ?

Exactly. It is based on an analysis of the ENTSO-E TP aggregated generation time-series. Max* is the maximum observed value for ramping up/down, q999 is the 99.9 percentile.

What is "link" in your generator modelisation ?

I use links both to model the reservoir hydropower plants and the cross-border interconnectors.

Regarding "other_map", do these correspond to generators that need to be added because the capacity given by entsoe is insufficient?

Unfortunately, ENTSO-E TP data sometimes is a bit sloppy and the generation can be misclassified as Other. The dictionary is then used to "redistribute" the capacity under Other to other types. This part should be revised.

Last question, could you explain me how you modelise Hydro dam and STEP (pump-turbine station)?

Start from here to see how to model storages into PyPSA.

[nribei]

Great, thanks ! I am on it :)