search

GFDRR / thinkhazard

ThinkHazard!
http://thinkhazard.org/
GNU General Public License v3.0
32 stars 19 forks source link

Issue while harvesting documents #747

Closed fvanderbiest closed 7 years ago

fvanderbiest commented 7 years ago

While harvesting newer labs geonode, we noticed these kind of errors:

WARNING document 792: hazard_type not supported: volcanic_ash,
WARNING document 831: hazard_type not supported: climate_change
WARNING document 498: hazard_type not supported: wind_storm
WARNING document 830: hazard_type not supported: sea_level_rise
WARNING document 830: hazard_type not supported: climate_change
WARNING document 681: hazard_type not supported: storm_surge
WARNING document 795: hazard_type not supported: landslide,
WARNING document 1010: hazard_type not supported: Drought
WARNING document 1010: hazard_type not supported: River Flood
WARNING document 1010: hazard_type not supported: Tsunami
WARNING document 1010: hazard_type not supported: Coastal Flood
WARNING document 1010: hazard_type not supported: Strong Wind
WARNING document 1010: hazard_type not supported: Volcanic Ash
WARNING document 1010: hazard_type not supported: Landslide

Issue comes from the fact that, eg in http://www.geonode-gfdrrlab.org/api/documents/792/ we have supplemental_information field with value "strong_wind, river_flood, drought, landslide, volcanic_ash,"

We should harden TH so that it does not fail on https://github.com/GFDRR/thinkhazard/blob/master/thinkhazard/processing/harvesting.py#L321

stufraser1 commented 7 years ago

Agreed, how best to resolve this?

fvanderbiest commented 7 years ago

Working on a regexp...

fvanderbiest commented 7 years ago

But document 1010 supplemental_information is clearly invalid:

WARNING document 1010: hazard_type not supported: Drought
WARNING document 1010: hazard_type not supported: River Flood
WARNING document 1010: hazard_type not supported: Tsunami
WARNING document 1010: hazard_type not supported: Coastal Flood
WARNING document 1010: hazard_type not supported: Strong Wind
WARNING document 1010: hazard_type not supported: Volcanic Ash
WARNING document 1010: hazard_type not supported: Landslide
stufraser1 commented 7 years ago

We can also make our data entry more robust. Once the new list view is complete, we'll be able to check and correct all that are causing problems.

fvanderbiest commented 7 years ago

Related PR is https://github.com/GFDRR/thinkhazard/pull/748

fvanderbiest commented 7 years ago

Issue can be closed if you agree.