gbif / portal-feedback

User feedback for the GBIF API, website and published data. You can ask questions here. 🗨❓
30 stars 16 forks source link

Administrative areas (gadm.org) outdated #3068

Open gbif-portal opened 4 years ago

gbif-portal commented 4 years ago

Administrative areas (gadm.org) outdated

We noticed you added a new field (administrative areas) to the filter options. A great initiative, but unfortunately the administrative divisions are outdated. The Netherlands f.i. now consist of 355 municipalities with often completely different names then the 491 municipalities still in use by gadm.org. Our records fully follow the current municipal system but are not being found by this new filter. Example: Dutch municipality "Lansingerland" generates 1804 records in our database, but 0 through Administrative areas..... same applies to 1085 records of municipality "Eijsden-Margraten". Please advise GADM to update their list of Dutch municipalities.


User: See in registry System: Firefox 82.0.0 / Windows 10.0.0 Referer: https://www.gbif.org/occurrence/search?institution_code=NMR Window size: width 1920 - height 938 API log&_a=(columns:!(_source),index:'prod-varnish-',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'response:%3E499')),sort:!('@timestamp',desc))) Site log&_a=(columns:!(_source),index:'prod-portal-',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'response:%3E499')),sort:!('@timestamp',desc))) System health at time of feedback: OPERATIONAL

jlegind commented 3 years ago

GADM.org was contacted through their contact form. http://rasterra.com/contact/gadm_contact_form

MortenHofft commented 3 years ago

See also https://github.com/gbif/hosted-portals/issues/123

MattBlissett commented 3 years ago

We are considering using GeoBoundaries instead of GADM. (Just an initial investigation at this point.) The Dutch boundaries are certainly different, with different borders and the inclusion of some sea/esturies.

https://www.geoboundaries.org/geoContrast.html?country=NLD&mainSource=geoBoundaries+%28Open%29&comparisonSource=GADM+v3.6&mainLevel=2&comparisonLevel=2

langeveldNMR commented 3 years ago

Thanks for suggesting GeoBoundaries. For The Netherlands this dataset is fine and represents current subdivisions.

abubelinha commented 2 years ago

I have just seen this new (for me) filter in gbif portal. From this related comment, looks like this search field is created by GBIF using coordinates taken from data provider.

But I wonder if the other-way round is possible.
I mean, is it now possible for data providers to fill in gadm values in IPT, so that this will be the value shown in portal? Or perhaps providing a polygon footprintWKT+footprintSRS of that Area? (instead of point lat/long coordinates).

Related: would this permit that occurrence records without coordinates show up in geospatial queries by means of their gadm values? So if somebody searches in gbif portal using a polygon or boundingbox, the occurrence records with overlapping gadm boundaries will show un in the query

Thanks @abubelinha

abubelinha commented 2 years ago

Related to previous questions, something else comes to my mind.

If filling gadm directly by data-proviers is not possible (in IPT), I still can figure out an interesting possibility for records without coordinates.

What if GBIF can use a table provided by data providers willing to do so, for interpreting info present in that provider's StateProvince and/or Municipality IPT fields? Example: CountryCode StateProvince Municipality gadm.org
PT Ag - PRT.9_1
PT Ag Vila do Bispo PRT.9.15_1
... ... ... ...

This provides way to fill in to a couple of ADM levels. So I think it could be pretty useful for showing in map searches a lot of records without coordinates (in case gbif is willing to do the polygon overlapping I asked above).

Of course this would be something optional. So if a given data provider wants to give you such a mapping table, it could be done by provinding csv data inside some particular IPT field, by means of a github linked file, or whatever.

dgasl commented 2 years ago

I am also interested in this as a possible workaround for occurrences without coordinates or even without verbatimLocality, but belonging to a dataset which comes from a well known administrative area.

Would a rough footprintWKT (which fits inside a given administrative area) be enough to make GBIF detect the Area, and attach it to these occurrences so it can be used for portal/api searches?

Thanks

abubelinha commented 5 months ago

@MattBlissett

We are considering using GeoBoundaries instead of GADM. (Just an initial investigation at this point.)

Any progress on switching to GeoBoundaries?
I can confirm Spanish GADM level 3 boundaries are also wrong. They don't follow Spanish "Municipios", which are the official ADM3 level for Spain (which GeoBoundaries correctly uses, BTW): https://en.wikipedia.org/wiki/Municipalities_of_Spain

If you browse Spanish GADM level 3 divisions with GBIF geocode API, they are called "comarcas", but their names are missing for the most part of Spain (just the west coastal areas are named, the others being named by numbers which nobody uses: 'n.a. (xxx)'). Their boundaries may or may not be correct (in NW Spain they are totally wrong, not even resembling any well-know division ... I wonder where GADM got that from), but that's not the point. The point is Spain is not officially divided into comarcas: only some Autonomous communities (ADM1) use that internal "comarcal" subdivision, but it is a local sublevel in between national Provincias (ADM2) and national Municipios (ADM3).

That said, I wouldn't mind using comarcal subdivision if GADM version wasn't so buggy. But even if it was correct, it is a higher level than the official ADM3 so I wonder how would you call that layer. ADM 2.5? Would you add ADM4 for Spain to include Municipios? (which would be incorrect since they are ADM3, I think). And the most important thing IMHO: Municipios are the smallest of all these levels, so they would be the most accurate informative names GBIF can provide for a coordinate-georreferenced point in Spain. So it is a big lost not showing them.

So I would definitely say "yes, go ahead with GeoBoundaries", or at least remove current GADM 3rd level for Spain, where it is a wrong useless layer in most parts of the country (at the very least, all those "comarcas" showing 'n.a. (xxx)' names).

abubelinha commented 3 months ago

Nobody answering here, but I keep interested. FYI geoboundaries.org has now also a github repository: https://github.com/wmgeolab/geoBoundaries/releases