codice / ddf

DDF Distributed Data Framework - an open source, modular integration framework.
http://ddf.codice.org
Other
132 stars 181 forks source link

[2.19.x] G-10122 Fix buffering across the antimeridian #6690

Closed jrnorth closed 2 years ago

jrnorth commented 2 years ago

What does this PR do?

Who is reviewing it?

@jlcsmith @kcwire @millerw8

Select relevant component teams:

@codice/solr

How should this be tested?

  1. Run gazetteer:update with https://github.com/codice/ddf/blob/03be912e331024f3bc6856d5e87f8e3b7db0b392/distribution/ddf-common/src/main/resources/data/countries.geo.json
  2. Run gazetteer:build-suggester-index
  3. Ingest the following files: russia1.json
    {
    "type": "Feature",
    "geometry": {
    "type": "Point",
    "coordinates": [-179.9375, 69.116389]
    },
    "properties": {
    "title": "Russia 1"
    }
    }

    russia2.json

    {
    "type": "Feature",
    "geometry": {
    "type": "Point",
    "coordinates": [179.900833, 71.637778]
    },
    "properties": {
    "title": "Russia 2"
    }
    }

    These locations are both within a 25km buffer of the Russia polygon, close to the antimeridian.

  4. Open Intrigue and create an anyGeo query with the keyword 'Russia'. Run it and verify you get no results.
  5. Add a buffer of 25km and verify you get both results.

Checklist:

Notes on Review Process

Please see Notes on Review Process for further guidance on requirements for merging and abbreviated reviews.

Review Comment Legend:

jrnorth commented 2 years ago

build now

cxddfbot commented 2 years ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 2 years ago

Build SUCCESS See the job results in legacy Jenkins UI or in Blue Ocean UI.

jlcsmith commented 2 years ago

Should this be forward ported to master?

jlcsmith commented 2 years ago

Hero successful

kcwire commented 2 years ago

With the addition of the normWrapLongitude to the Solr schema, will a full reindex of any data that might be impacted be required?

jrnorth commented 2 years ago

@kcwire

With the addition of the normWrapLongitude to the Solr schema, will a full reindex of any data that might be impacted be required?

That's a good question. I would think a reindex would be required, since any records with geometries outside that range should have failed to be indexed in the first place without normWrapLongitude enabled, but I can do some testing to verify that.

jrnorth commented 2 years ago

@jlcsmith

Should this be forward ported to master?

I think so. The normWrapLongitude change to the Solr schema has already been in master for a while, but the changes to SolrFilterDelegate are new.

jrnorth commented 2 years ago

@kcwire ~Looks like records outside that range are ingested successfully, they are just marked as invalid.~ Turns out I mistakenly had normWrapLongitude enabled when I tested that again. Records outside [-180, 180] are not ingested, so a reindex is required.

jrnorth commented 2 years ago

build now

cxddfbot commented 2 years ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 2 years ago

Build SUCCESS See the job results in legacy Jenkins UI or in Blue Ocean UI.