search

CatalogueOfLife / backend

Complete backend of COL ChecklistBank
Apache License 2.0
15 stars 11 forks source link

Cannot re-sync sectors in cancelled state #1295

Closed gdower closed 6 months ago

gdower commented 7 months ago

@mdoering, when you cancelled the sector syncs over the [sic] bug, it seems to have broken the sync queue. It's not possible to re-sync these four datasets with sectors in cancelled state:

MilliBase, Mollusca, Isopoda, Bryozoa

I get a 201 on the sync POST, but it doesn't seem to sync. The other WoRMS sectors with syncs that weren't in cancelled state seemed to have worked.

Logs&_a=(columns:!(level,sectorKey,service,logger_name,message),filters:!()))

java.lang.InterruptedException: java.lang.NullPointerException
    at life.catalogue.common.lang.InterruptedRuntimeException.asChecked(InterruptedRuntimeException.java:24)
    at life.catalogue.assembly.SectorSync.processTree(SectorSync.java:320)
    at life.catalogue.assembly.SectorSync.doWork(SectorSync.java:95)
    at life.catalogue.assembly.SectorRunnable.run(SectorRunnable.java:136)
    at life.catalogue.assembly.SectorSync.run(SectorSync.java:44)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
mdoering commented 7 months ago

This is very weird. Im investigating...

mdoering commented 7 months ago

btw, the logs also say this:

Rematched 60 SpeciesEstimates from project 3. updated: 1, broken: 59

mdoering commented 7 months ago

There was a bug in the code that swallowed some exceptions and converted them into interrupted exceptions, used to cancel operations. So there is some other, real exception underneath this problem that we cannot see so far. I will deploy the fix in an hour and try to sync millibase again, so we can hopefully see and ultimately fix the real problem

mdoering commented 7 months ago

deployed. Finally we can ee a NullPointerException

Caused by: java.lang.NullPointerException: null at life.catalogue.assembly.TreeBaseHandler.processEnd(TreeBaseHandler.java:169) at life.catalogue.assembly.TreeCopyHandler.process(TreeCopyHandler.java:164) at life.catalogue.assembly.TreeCopyHandler.processLast(TreeCopyHandler.java:125) at life.catalogue.assembly.TreeCopyHandler.acceptThrows(TreeCopyHandler.java:115)

mdoering commented 7 months ago

It was introduced by the sic parsing in authorships. Datasets with sic synonyms currently fail. I might need a day to fix this, its a bit more complex I am afraid.

mdoering commented 6 months ago

deployed to prod just now and synced millibase successfully