search

pypsa-meets-earth / pypsa-earth

PyPSA-Earth: A flexible Python-based open optimisation model to study energy system futures around the world.
https://pypsa-earth.readthedocs.io/en/latest/
225 stars 177 forks source link

power_data_extractor_tool unexpected lines #89

Closed davide-f closed 3 years ago

davide-f commented 3 years ago

When applying the power extractor tool to the whole africa about 100 unexpected lines are saved in the csv. Such lines may not have buses, or a specific country associated. In github I push the output file I obtained, which is the most updated, given the scripts

pz-max commented 3 years ago

We extract in the extractor tool all the OpenStreetMap data from a given set of countries. These files are almost not cleaned, hence considered as "raw". The cleaning script should make sure that these empty columns are removed. If this does not happen, we should make sure it removes such blanked lines.

Some stuff which might help:

davide-f commented 3 years ago

Yeah, I agree on the above, but there are some issues. Some points are far away in the ocean, thus they cannot be easily set in a specific country. That applies for several lines for example, which lead to warnings in build_bus_regions (I discovered it because of that)

pz-max commented 3 years ago

Do you ave a plot on these ocean buses? "Ocean" buses are not necessarily articial. Offshore wind plants are represented by such. I know that PyPSA-Eur had something like buses["offshore_bus"] = True, that probably dealt with that issue. Maybe we should do smt similar?

From: davide-f @.> Sent: 13 September 2021 17:25 To: pypsa-meets-africa/pypsa-africa @.> Cc: PARZEN Maximilian @.>; Comment @.> Subject: Re: [pypsa-meets-africa/pypsa-africa] power_data_extractor_tool unexpected lines (#89)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

Yeah, I agree on the above, but there are some issues. Some points are far away in the ocean, thus they cannot be easily set in a specific country. That applies for several lines for example, which lead to warnings in build_bus_regions (I discovered it because of that)

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/pypsa-meets-africa/pypsa-africa/issues/89#issuecomment-918362035, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOYZENLUFULZZRMOA75XNNLUBYQX3ANCNFSM5D55RV7A.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

davide-f commented 3 years ago

Exactly, the errors related to the lines are that no buses are linked to those lines. Regarding the buses, I checked the weird coordinates in the raw data. Now I'm fixing other stuff, as I have time I move there.

davide-f commented 3 years ago

I investigated and the output lines to plot are ok, but the main concerns are in the input data. I think we may have to better investigate the raw input data. For example, no lines are found for Namibia, but if I look at the file NA_lines.json there are several lines. I am relatively new to the last details of data_extractor (I worked on data cleaning months ago), which is the reason I am highlighting this at a high-level, but maybe some of the missing data that we are experiencing may be solved by investigating that. P.S. I added some plotting features of the lines in the notebook of bus_regions What is your feeling?

pz-max commented 3 years ago

My feeling is that it’s worth revising the osm_build_network, cleaning an du extraction tool. I also shared yesterday some issue on GitHub describing what goes wrong

Get Outlook for iOShttps://aka.ms/o0ukef

From: davide-f @.> Sent: Monday, September 13, 2021 7:25:54 PM To: pypsa-meets-africa/pypsa-africa @.> Cc: PARZEN Maximilian @.>; Comment @.> Subject: Re: [pypsa-meets-africa/pypsa-africa] power_data_extractor_tool unexpected lines (#89)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

I investigated and the output lines to plot are ok, but the main concerns are in the input data. I think we may have to better investigate the raw input data. For example, no lines are found for Namibia, but if I look at the file NA_lines.json there are several lines. I am new to the details of data_extractor, which is the reason I am highlighting this at a high-level, but maybe some of the missing data that we are experiencing may be solved by investigating that. P.S. I added some plotting features of the lines in the notebook of bus_regions What is your feeling?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/pypsa-meets-africa/pypsa-africa/issues/89#issuecomment-918461733, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOYZENKNSYCAE75UVLSKORTUBY63FANCNFSM5D55RV7A.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

davide-f commented 3 years ago

This issue has been mainly solved by #91; however there are still some smaller inaccuracies. See the plots using the new notebook build_bus_regions_verifybuseslines in the notebook folder. Now only few lines are not matched: still some work is needed but the main problem (related to Namibia) is solved.

davide-f commented 3 years ago

I close this issue as it has been solved by #96