search

AtlasOfLivingAustralia / biocache-store

Occurrence processing, indexing and batch processing
Other
7 stars 24 forks source link

DatePrecisionTest functions different in debugger than out of debugger #289

Closed ansell closed 5 years ago

ansell commented 5 years ago

The DatePrecisionTest functions different in a debugger than out of a debugger. While testing date refactoring, unable to replicate the results of a test failure outside of the debugger compared to inside it. Opening this as a tracking issue, although it may still be related to the refactoring.

Outside of the debugger the following unexpected canonical values for datePrecision are found for the 'y' test, but not for the 'year' test:

 2018-10-22 16:02:31,361 INFO : [EventProcessor] - raw datePrecision was: D, Canonical datePrecision was: Day
 2018-10-22 16:02:31,371 INFO : [EventProcessor] - raw datePrecision was: d, Canonical datePrecision was: Day
 2018-10-22 16:02:31,377 INFO : [EventProcessor] - raw datePrecision was: day, Canonical datePrecision was: Day
 2018-10-22 16:02:31,382 INFO : [EventProcessor] - raw datePrecision was: Day, Canonical datePrecision was: Day
 2018-10-22 16:02:31,386 INFO : [EventProcessor] - raw datePrecision was: O, Canonical datePrecision was: Month
 2018-10-22 16:02:31,396 INFO : [EventProcessor] - raw datePrecision was: Month, Canonical datePrecision was: Month
 2018-10-22 16:02:31,401 INFO : [EventProcessor] - raw datePrecision was: M, Canonical datePrecision was: Month
 2018-10-22 16:02:31,404 INFO : [EventProcessor] - raw datePrecision was: Y, Canonical datePrecision was: Before Year
 2018-10-22 16:02:31,418 INFO : [EventProcessor] - raw datePrecision was: y, Canonical datePrecision was: Before Year
 2018-10-22 16:02:31,425 INFO : [EventProcessor] - raw datePrecision was: year, Canonical datePrecision was: Year
 2018-10-22 16:02:31,435 INFO : [EventProcessor] - raw datePrecision was: YEAR, Canonical datePrecision was: Year
 2018-10-22 16:02:31,446 INFO : [EventProcessor] - raw datePrecision was: YY, Canonical datePrecision was: Year Range

The code added to EventProcessor.checkPrecision to generate this output was the following. The tests fail with or without the logging:

        logger.info("raw datePrecision was: " + raw.event.datePrecision + ", Canonical datePrecision was: " + term.canonical)

If anyone has any experience with this code to shed light on where to follow up debugging the unexpected Before Year value it would be great, as that value doesn't appear to be in any of the source files for biocache-store.

djtfmartin commented 5 years ago

I have a recollection of adding something for "Before Year", and then removing it because I was unclear on the semantics (its a value in use by NBN UK only i think). I cant see a test that references it.

The tests all pass for me (and travis). Sorry if ive missed a detail here...

ansell commented 5 years ago

I figured out the behaviour difference. In the debugger, the datePrecision.txt file from https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/resources/datePrecision.txt was used, as expected. Outside of the debugger, an old file deployed by ansible/ala-install at some point in the past to /data/biocache/config/vocab/dataPrecision.txt on the file system was being used, which did have Before Year in it.

I am not familiar with the way any of the run configurations or biocache-store tests (or Scala in general) choose between the file system and the classpath, but this may happen again to someone in the future. Closing this now, but will reopen if I figure out what the bug in the choice mechanism may be so at least everything runs the same on a single computer in and out of debugger mode.

To clarify, this wasn't on the master branch, this is on my branch for date refactoring.

ansell commented 5 years ago

The commit where Before Year was removed was relatively recent in July https://github.com/AtlasOfLivingAustralia/biocache-store/commit/0398df452abd311a0463c6d4a4b4e8449bc93f6d