search

ilovesoup / hyracks

Automatically exported from code.google.com/p/hyracks
Apache License 2.0
0 stars 0 forks source link

BTree upsert throws duplicate key exception #122

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Theoretically this trace should never happen, but it happens in Pregelix job on 
a cluster.

Caused by: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.
at 
edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.findInsertTupleInd
ex(BTreeNSMLeafFrame.java:75)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.updateLeaf(BTree.java:478)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.upsertLeaf(BTree.java:499)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:727)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)
at 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java
:286)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:331)
at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$500(BTree.java:68)
at 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfCondition
ElseInsert(BTree.java:895)
at 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java
:888)
I wonder why this could ever happens...

Original issue reported on code.google.com by buyingyi@gmail.com on 29 Jul 2013 at 2:34

GoogleCodeExporter commented 9 years ago 
This is another trace, but in the update call:

edu.uci.ics.hyracks.storage.am.btree.exceptions.BTreeException: 
java.lang.IllegalStateException: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:783)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java
:286)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.update(BTree.java:348)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$300(BTree.java:68)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.update(BTree.java
:877)\n\tat 
edu.uci.ics.pregelix.dataflow.util.UpdateBuffer.updateIndex(UpdateBuffer.java:10
1)\n\tat 
edu.uci.ics.pregelix.dataflow.util.CopyUpdateUtil.copyUpdate(CopyUpdateUtil.java
:49)\n\tat 
edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOpe
ratorNodePushable.writeResults(IndexNestedLoopRightOuterJoinFunctionUpdateOperat
orNodePushable.java:287)\n\tat 
edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOpe
ratorNodePushable.outputMatch(IndexNestedLoopRightOuterJoinFunctionUpdateOperato
rNodePushable.java:213)\n\tat 
edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOpe
ratorNodePushable.nextFrame(IndexNestedLoopRightOuterJoinFunctionUpdateOperatorN
odePushable.java:199)\n\t... 10 more\nCaused by: 
java.lang.IllegalStateException: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.\n\tat 
edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.split(BTreeNSMLeaf
Frame.java:198)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performLeafSplit(BTree.java:431
)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.updateLeaf(BTree.java:484)\n\ta
t 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:732)\n\t..
. 21 more\nCaused by: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.\n\tat 
edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.findInsertTupleInd
ex(BTreeNSMLeafFrame.java:75)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.split(BTreeNSMLeaf
Frame.java:196)

Original comment by buyingyi@gmail.com on 30 Jul 2013 at 8:06

GoogleCodeExporter commented 9 years ago 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.update  should 
never throw TreeIndexDuplicateKeyException.

Original comment by buyingyi@gmail.com on 30 Jul 2013 at 8:08

GoogleCodeExporter commented 9 years ago 
Here is another trace:

edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOpe
ratorNodePushable.nextFrame(IndexNestedLoopRightOuterJoinFunctionUpdateOperatorN
odePushable.java:208)\n\tat 
edu.uci.ics.pregelix.dataflow.MaterializingReadOperatorDescriptor$1.nextFrame(Ma
terializingReadOperatorDescriptor.java:63)\n\t... 9 more\nCaused by: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.\n\tat 
edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.findInsertTupleInd
ex(BTreeNSMLeafFrame.java:75)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.updateLeaf(BTree.java:478)\n\ta
t 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:732)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java
:286)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.update(BTree.java:348)\n\tat 
edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$300(BTree.java:68)\n\tat

edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.update(BTree.java
:877)\n\tat 
edu.uci.ics.pregelix.dataflow.util.UpdateBuffer.updateIndex(UpdateBuffer.java:10
1)\n\tat 
edu.uci.ics.pregelix.dataflow.util.CopyUpdateUtil.copyUpdate(CopyUpdateUtil.java
:49)\n\tat 
edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOpe
ratorNodePushable.writeResults(IndexNestedLoopRightOuterJoinFunctionUpdateOperat
orNodePushable.java:287)\n\tat

Original comment by buyingyi@gmail.com on 31 Jul 2013 at 12:20

GoogleCodeExporter commented 9 years ago 
Run with one partition per machine, and can still hit this:

Caused by: 
edu.uci.ics.hyracks.storage.am.common.exceptions.TreeIndexDuplicateKeyException:
 Trying to insert duplicate key into leaf node.
    at edu.uci.ics.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.findInsertTupleIndex(BTreeNSMLeafFrame.java:75)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.updateLeaf(BTree.java:478)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:732)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:630)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:286)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.update(BTree.java:348)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$300(BTree.java:68)
    at edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.update(BTree.java:877)
    at edu.uci.ics.pregelix.dataflow.util.UpdateBuffer.updateIndex(UpdateBuffer.java:101)
    at edu.uci.ics.pregelix.dataflow.util.CopyUpdateUtil.copyUpdate(CopyUpdateUtil.java:49)
    at edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.writeResults(IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java:287)
    at edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.outputMatch(IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java:213)
    at edu.uci.ics.pregelix.dataflow.std.IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.nextFrame(IndexNestedLoopRightOuterJoinFunctionUpdateOperatorNodePushable.java:199)

Original comment by buyingyi@gmail.com on 31 Jul 2013 at 8:57

GoogleCodeExporter commented 9 years ago 
Providing a way to reproduce the issue from my end would help.
If that's not possible, then let's look it at together today.

Sattam

Original comment by salsuba...@gmail.com on 31 Jul 2013 at 1:15

GoogleCodeExporter commented 9 years ago 
It turned out the data is being loaded with duplicates (checking for duplicates 
were omitted), causing the index to be corrupted.

Original comment by salsuba...@gmail.com on 1 Aug 2013 at 12:58