Open Mahir-Sparkess opened 2 years ago
The ESGF lon and lat coordinates range from -90
to 90
for latitude and 0
to 360
for longitude.
This breaks elasticsearch geoshape as that requires longitude range of -180
to 180
Need to update the assets/items/collections so longitude values are shifted by 180 for ["properties.min_lon", "properties.max_long", "spatial.bbox"]
The min and max lon properties are wrong, they need to be switched. currently min_lon
represents east degrees and max_lon
represents west degrees.
There are too many nuances using the cardinal degrees to represent latitude and longitude. It might be worth reindexing with a different method of ingesting the bbox:
The ESGF Solr datasets have a geo/geos field with spatial information that uses correct ranges, however it can be a string or a list of strings in the format: ENVELOPE (min_lon, max_lon, max_lat, min_lat)
. Will need to look up how to convert a geos field to an appropriate bbox. The geos field can have one or many geo_shape string that have different values but seem to describe the same area. It is not standardised accross all the datasets.
geos
so if there is no geos field, use the cardinal degrees to add min_lat, max_lat, min_lon and max_lon to the asset properties? (the current method)[minLon, maxLon, maxLat, minLat]
(This change might not be necessary, it seems elasticsearch can read what the current order is anyways.)When redoing the index, ensure the mapping for the new index for spatial.properties.bbox.properties.type = "geo_shape"
, it should be the case in the breezy deploy anyways: https://breezy.badc.rl.ac.uk/mrahman/stac-esgf-indexer-deploy/-/blob/master/images/item-generator/files/item-mapping.yml
Current bbox search throws this error:
pystac_client.exceptions.APIError: {"detail":"RequestError(400, 'search_phase_execution_exception', 'failed to find type for field [spatial.bbox]')"}
bbox is now detected as a geo_shape and is in the field
spatial.bbox
but the mapping for it is as follows:The mapping needs to be updated to as such stated here:
Since elasticsearch doesn't support updating the mapping of an existing field, it will require a reindex.
TO-DO:
min_lon
andmax_lon
values of docs in the esgf index for assets, items and collections.Shift the values ofmin_lon
andmax_lon
by -180 and update thebbox
. There exists two items in which the bbox values are correct:892cfaee29239415d6b61b0fb027877e
,1b6b26545b55b0e6125b78d155881e50
Update the esgf-solr-input plugin to swap the values of west and east degrees when constructing the bbox at asset generation.