search

openego / data_processing

(geo)data processing, database setup, data validation
GNU Affero General Public License v3.0
7 stars 5 forks source link

Some generators have 2*8760 values for p_max_pu & p_set #220

Open IlkaCu opened 6 years ago

IlkaCu commented 6 years ago

As reported in the eTraGo repository some generators located in neighbouring countries have 2*8760 values for p_max_pu & p_set. This bug occurs in the DP results v0.3.0.

IlkaCu commented 6 years ago

In v0.3.0 this occurs 24 times in total and differs between the three scenarios. But only wind power plants (source = 13) are affected. grafik

IlkaCu commented 6 years ago

@s3pp: Do you have an idea how this happens? The only thing which seems a bit odd from my point of view is one DELETE statement which is commented out.

eosram commented 6 years ago

I agree, it seems odd. I will further investigate.

jankaeh commented 6 years ago

Am I right, that for p_set there are more cases (including 3*8760 values)?:

Generators with 26280 values for p_set (only Status Quo): ['200001', '200002', '200003', '200004', '200005', '200006', '200007', '200008', '200009', '200010', '200011', '200012', '200013', '200014', '200015', '200016']

Generators with 17520 values for p_set (only Status Quo): ['200017', '200018', '200019', '200020', '200021', '200022', '200023', '200024', '200025', '200026', '200027', '200028', '200029', '200030', '200031', '200032', '200033', '200047', '200048', '200056', '200057', '200062', '200063']

IlkaCu commented 6 years ago

I investigated a bit further and @jankaeh is right. In v0.3.0 I found in total 106 generators (39 in the Status Quo scenario) which have more than 8760 values for p_set. This occurs also for generators with a source other than 13 (wind). This bug occurs in the older versions of the data set (v0.2.10 and v0.2.11) as well.

eosram commented 6 years ago

Just fyi I had to make a local copy of the relevant tables. I do not want to temper with the present state of the model draft. Hopefully I can give a meaningful update or present a fix soon.

jankaeh commented 6 years ago

I think there are 52 Generators in v0.3.0 Status Quo without values for p_set. Might that be related to this issue? Maybe the second (and third) set of 8760 values of the generators with to many values for p_set should be addressed to one of these generators? What stands against this idea is, that there are 54 sets of 8760 values to much on the one hand and 52 missing sets on the other. But it's close ;-)...

52 generators without values for p_set: Index(['23073', '23647', '24016', '26584', '26911', '27095', '27588', '30388', '32391', '34122', '34692', '34984', '35582', '35583', '35608', '35617', '35638', '35639', '35653', '35684', '35694', '35714', '35735', '35736', '35745', '35768', '35800', '35805', '35840', '35850', '35855', '23004', '24419', '25213', '26126', '26847', '28512', '28677', '28807', '28859', '29615', '30356', '30853', '31100', '32309', '33248', '33434', '34087', '34879', '35602', '35603', '36122'], dtype='object')

IlkaCu commented 6 years ago

@s3pp: Any news here?

eosram commented 6 years ago

Thanks @jankaeh for your insights! I think this problem was introduced by the scenario files / renpassG!S results having data on wind on- AND offshore, but timeseries generator only handled wind as one source. Without further thought I reimplemented the script in python. Unfortunately it took a while, so I guess, this is not up to date with the newest developments. Maybe @IlkaCu you could point me in the right direction, what I most likely have missed in the past weeks of development, so I could adapt the pending pull request accordingly before your review.