Bounding box filter on step 1 not bringing all the collections

[evacougnon]

This issue tracker is only for AODN Portal issues.

Steps to reproduce the issue

1. Step 1 of Portal systest https://portal-systest.aodn.org.au/search
2. Expend the "Geographic Boundary" filter
3. Selected a bounding box just off Davis station in Antarctica (est of the Amery ice shelf, or south est of the Kerguelen plateau -- see screenshot below)

Expected behaviour

List all the collections with data matching the bounding box selected in the filters

Actual behaviour

The results give at least 3 collections without data for the selected area:

• IMOS - SOOP-Sea Surface Temperature (SST) Sub-facility - Near real-time data ; even the spatial extend shown on step 1 doesn't show anything in this area
• CSIRO O&A RV Investigator XBT Data Overview (2014 onwards) ; spatial extend on step 1 indicates there might be something but no data available in this area in step 2
• CSIRO O&A Current Meter Mooring Data Overview (1983 - 1998) ; spatial extend on step 1 indicates there might be something but no data available in this area in step 2 At least one collection missing:
• IMOS - Animal Tracking Facility - Satellite Relay Tagging Program - Near real-time data with quality-controlled locations -- potentially linked to https://github.com/aodn/PO-Backlog/issues/3432 but weird it only impacts the NRT collection while the DM looks fine and they're both using teh same harvester

Screenshots.

If the issue is related to the GUI, screenshots can be added to this issue via drag & drop.

What version of the aodn portal are you using?

4.42.102

Which browser are you using?

Chrome

Any extra information that might be useful in the debugging process.

This is normally the contents of a debug.log or config.log file. Raw text or a link to a pastebin type site are preferred. Associated test: https://aodn.testrail.com/index.php?/tests/view/84329

PR @craigrose

DOD @vietnguyengit

[Chetnamann]

same issue

What version of the aodn portal are you using? 4.42.104

Which browser are you using? Chrome

[vfisaac]

Feedback from Nat - these 2 external collections can be ignored due to the spatial extent in their metadata records

• CSIRO O&A RV Investigator XBT Data Overview (2014 onwards)
• CSIRO O&A Current Meter Mooring Data Overview (1983 - 1998)

[atkinsn]

The issue with the SOOP SST collection, is that for some reason Geonetwork is combining all the spatial extents into 1 bounding box, hence the false results.

Contrast with another collection which is maintaining its accurate spatial extents in Geonetwork full view IMOS - SOOP Expendable Bathythermographs (XBT) Research Group - XBT real-time data

[utas-raymondng]

Side note, if you click the more button, it seems point to the wrong page? more

[atkinsn]

Side note, if you click the more button, it seems point to the wrong page? more

Weird - its the right metadata record, as the title is correct, however the Point of truth URL points to one of its sister records (also in the Portal). I will delete it, and re-harvest.

[atkinsn]

CSIRO have the wrong point of truth URL on this record, I will reach out to them to amend

[atkinsn]

CSIRO have fixed it :)

[utas-raymondng]

investigation notes on how to repeat the case:

How to re-produce

1. You run a genetwork instance on your local PC. https://github.com/aodn/geonetwork-build
2. Export the XML from catalogue-imos or other geonetwork of this UUID ca15915b-38c5-4e95-8d25-9cc42c4dd485, attached file catalog-imos.zip
3. Run this url&fast=index)
4. You see this appear in the output XML

   <geonet:info xmlns:geonet="http://www.fao.org/geonetwork">
   <id>130</id>
   <uuid>ca15915b-38c5-4e95-8d25-9cc42c4dd485</uuid>
   <schema>iso19115-3.2018</schema>
   <createDate>2023-08-11T16:20:54</createDate>
   <changeDate>2023-08-14T15:39:01</changeDate>
   <source>54f007c1-9245-41ff-839e-dfc1980275b5</source>
   <isPublishedToAll>true</isPublishedToAll>
   <view>true</view>
   <notify>false</notify>
   <download>true</download>
   <dynamic>true</dynamic>
   <featured>false</featured>
   <selected>false</selected>
   <category internal="true">dataset</category>
   </geonet:info>

   Which is not expected

Why geonetwork return this

I have no idea at the moment, the problem goes down to this particular class, so if you set a break point to this line for class org.locationtech.jts.geom.Envelope method

public boolean intersects(Envelope other) {
    if (!this.isNull() && !other.isNull()) {
        return !(other.minx > this.maxx) && !(other.maxx < this.minx) && !(other.miny > this.maxy) && !(other.maxy < this.miny);
    } else {
        return false;
    }
}

The first return line return true for some of the bounding box area namely (a) Env[minx = 87.890625 : maxx = 97.20703125, miny = -67.984375 : maxy = -61.48046875], matches this (b) Env[minx = -180, maxx = 180, min y = -70, ,maxy = 50]. Where (b) comes I have no clue. (a) comes from the request link query parameter geometory=POLYGON((87.890625 -61.48046875,97.20703125 -61.83203125,95.44921875 -67.984375,90.17578125 -67.10546875,87.890625 -61.48046875))

The geonetwork in this case use relation = overlaps_bbox class to do the compare. (OverlapsBBoxFilter.class)

The calling class is SearchManager.class and there it provision the filter class

types.put("encloses".toLowerCase(), SearchManager.constructor(ContainsFilter.class));
types.put("crosses".toLowerCase(), SearchManager.constructor(CrossesFilter.class));
types.put("fullyOutsideOf".toLowerCase(), SearchManager.constructor(IsFullyOutsideOfFilter.class));
types.put("equal".toLowerCase(), SearchManager.constructor(EqualsFilter.class));
types.put("intersection".toLowerCase(), SearchManager.constructor(IntersectionFilter.class));
types.put("overlaps".toLowerCase(), SearchManager.constructor(OverlapsFilter.class));
types.put("touches".toLowerCase(), SearchManager.constructor(TouchesFilter.class));
types.put("within_bbox".toLowerCase(), SearchManager.constructor(WithinBBoxFilter.class));
types.put("overlaps_bbox".toLowerCase(), SearchManager.constructor(OverlapsBBoxFilter.class));

and the filter happens in this method, and finally call intersects in the Environment -> intersects()

public SpatialFilter filter(Query query, int numHits, Collection<Geometry> geom, Element request) throws Exception {
            this._lock.lock();

            SpatialFilter var6;
            try {
                String relation = Util.getParam(request, "relation", "intersection");
                if (geom.size() != 1) {
                    Collection<SpatialFilter> filters = new ArrayList(geom.size());
                    Envelope bounds = null;

                    Geometry geometry;
                    for(Iterator var8 = geom.iterator(); var8.hasNext(); filters.add(((Constructor)this._types.get(relation)).newInstance(query, numHits, geometry, new SearchManager.Spatial.SpatialIndexAccessor()))) {
                        geometry = (Geometry)var8.next();
                        if (bounds == null) {
                            bounds = geometry.getEnvelopeInternal();
                        } else {
                            bounds.expandToInclude(geometry.getEnvelopeInternal());
                        }
                    }

[utas-raymondng]

@ggalibert mentioned that it maybe a low priority task given portal on demise path.