search

AdamaJava / adamajava

Other
14 stars 5 forks source link

q3tiledaligner - synchronise loading of reference to avoid race conditions #238

Closed holmeso closed 3 years ago

holmeso commented 3 years ago

Describe the bug One of the external libraries that q3tiledaligner uses (trove4j) is complaining that it has objects in its map that are equal but have different hashCodes, which violates the hashCode contract.

It looks like this is because there is a race condition to load a contig into a shared cache.

The addition of the synchronized keyword to the method/block that loads this into the shared cache should clean this up.

To Reproduce

Unfortunately, to reproduce this error is hard as it requires more than 1 thread to attempt to load the same bit of reference into the shared cache at the same time.

Expected behavior The reference for a contig should be able to be added to the shared cache without error.

Error log

16:13:56.330 [pool-89-thread-1] SEVERE org.qcmg.qsv.softclip.FindClipClustersMT - Setting exit status in clip cluster thread to 1 as exception caught: java.lang.IllegalArgumentException: Equal objects must ha
ve equal hashcodes. During rehashing, Trove discovered that the following two objects claim to be equal (as in java.lang.Object.equals()) but their hashCodes (or those calculated by your TObjectHashingStrateg
y) are not equal.This violates the general contract of java.lang.Object.hashCode().  See bullet point two in that method's documentation. object #1 =class java.lang.String id= 384333531 hashCode= -2126244324
toString= /reference/genomes/GRCh37_ICGC_standard_v2/indexes/BWAKIT_0.7.12/GRCh37_ICGC_standard_v2.fachr8; object #2 =class java.lang.String id= 384333531 hashCode= -2126244324 toString= /reference/genomes/GR
Ch37_ICGC_standard_v2/indexes/BWAKIT_0.7.12/GRCh37_ICGC_standard_v2.fachr8
Inserting same object twice, rehashing bug. Object= /reference/genomes/GRCh37_ICGC_standard_v2/indexes/BWAKIT_0.7.12/GRCh37_ICGC_standard_v2.fachr8
        at gnu.trove.impl.hash.TObjectHash.buildObjectContractViolation(TObjectHash.java:464)
        at gnu.trove.impl.hash.TObjectHash.throwObjectContractViolation(TObjectHash.java:448)
        at gnu.trove.map.hash.THashMap.rehash(THashMap.java:403)
        at gnu.trove.impl.hash.THash.postInsertHook(THash.java:388)
        at gnu.trove.map.hash.THashMap.doPut(THashMap.java:178)
        at gnu.trove.map.hash.THashMap.put(THashMap.java:145)
        at au.edu.qimr.tiledaligner.util.ReferenceUtil.loadIntoReferenceCache(ReferenceUtil.java:51)
        at au.edu.qimr.tiledaligner.util.ReferenceUtil.getRefFromChrStartStop(ReferenceUtil.java:27)
        at au.edu.qimr.tiledaligner.util.TiledAlignerUtil.getRefFromChrPos(TiledAlignerUtil.java:1146)
        at au.edu.qimr.tiledaligner.util.TiledAlignerUtil.smithWaterman(TiledAlignerUtil.java:2169)
        at au.edu.qimr.tiledaligner.util.TiledAlignerUtil.getBlatRecordsSWAll(TiledAlignerUtil.java:1985)
        at au.edu.qimr.tiledaligner.util.TiledAlignerUtil.runTiledAlignerCache(TiledAlignerUtil.java:1192)
        at au.edu.qimr.tiledaligner.util.TiledAlignerUtil.runTiledAlignerCache(TiledAlignerUtil.java:1175)
        at org.qcmg.qsv.softclip.FindClipClustersMT$Clustering.blatBreakpoints(FindClipClustersMT.java:627)
        at org.qcmg.qsv.softclip.FindClipClustersMT$Clustering.defineClipPositions(FindClipClustersMT.java:585)
        at org.qcmg.qsv.softclip.FindClipClustersMT$Clustering.clusterSoftClips(FindClipClustersMT.java:505)
        at org.qcmg.qsv.softclip.FindClipClustersMT$Clustering.run(FindClipClustersMT.java:446)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)