search

ilovesoup / hyracks

Automatically exported from code.google.com/p/hyracks
Apache License 2.0
0 stars 0 forks source link

BTree Bulk Load generates "Page is pinned and file is being closed" Exception on TPCH 1x #8

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Nick's original E-Mail to me:

There is something else you can help us with, when you're done with your other 
project.

I wrote a little script which starts one Hyracks CC + 1 NC, deploys Asterix and 
then loads TPC-H 1x. Then shuts down Hyracks and repeats from the beginning. 
(The script is attached.)

While running this loop, at some point the BTreeBulkLoad generates the 
exception "Page is pinned and file is being closed".

So I think it was not related to using wrong names for datasets, but the load 
in general has a bug, which only manifests itself sporadically.

Original issue reported on code.google.com by alexande...@gmail.com on 23 Apr 2011 at 3:30

Attachments:

GoogleCodeExporter commented 9 years ago 
In the past I was able to reproduce this error by restarting Hyracks and doing 
a load in a loop. Now, even if I let the script loop for over half an hour, it 
doesn't throw the error. But, occasionally, when loading a B-tree, the buffer 
cache crashes with the following trace:

INFO: 
5aa9bec7-833b-4530-ae0e-e72414bdb0b3:e440783f-3472-4885-9618-2ecda9239026:ODID:5
81e0afd-f802-4dd0-81fc-0481d346d09f:2(OperatorRunnable[edu.uci.ics.hyracks.stora
ge.am.btree.dataflow.BTreeBulkLoadOperatorNodePushable@78878c4c]): ABORTED
java.lang.RuntimeException: java.lang.IllegalStateException: Page is pinned and 
file is being closed
        at edu.uci.ics.hyracks.control.nc.runtime.OperatorRunnable.run(OperatorRunnable.java:85)
        at edu.uci.ics.hyracks.control.nc.Stagelet$1.run(Stagelet.java:135)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalStateException: Page is pinned and file is being 
closed
        at edu.uci.ics.hyracks.storage.common.buffercache.BufferCache.invalidateIfFileIdMatch(BufferCache.java:559)
        at edu.uci.ics.hyracks.storage.common.buffercache.BufferCache.sweepAndFlush(BufferCache.java:539)
        at edu.uci.ics.hyracks.storage.common.buffercache.BufferCache.closeFile(BufferCache.java:578)
        at edu.uci.ics.hyracks.storage.am.btree.dataflow.BTreeOpHelper.deinit(BTreeOpHelper.java:134)
        at edu.uci.ics.hyracks.storage.am.btree.dataflow.BTreeBulkLoadOperatorNodePushable.close(BTreeBulkLoadOperatorNodePushable.java:88)
        at edu.uci.ics.hyracks.control.nc.runtime.OperatorRunnable.pushFrames(OperatorRunnable.java:102)
        at edu.uci.ics.hyracks.control.nc.runtime.OperatorRunnable.run(OperatorRunnable.java:78)
        ... 4 more

Original comment by nicola.o...@gmail.com on 21 Jun 2011 at 5:57

GoogleCodeExporter commented 9 years ago

Original comment by nicola.o...@gmail.com on 21 Jun 2011 at 5:58

GoogleCodeExporter commented 9 years ago 
Issue 18 has been merged into this issue.

Original comment by nicola.o...@gmail.com on 21 Jun 2011 at 5:59

GoogleCodeExporter commented 9 years ago 
In the stabilization branch r1463 I just noticed a similar exception during the 
mvn package:

Running edu.uci.ics.hyracks.storage.am.btree.multithread.BTreeMultiThreadTest
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: Failed to pin page 
0:675 because all pages are pinned.
        at edu.uci.ics.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:145)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:511)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:543)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:543)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:242)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insert(BTree.java:259)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$1000(BTree.java:56)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.insert(BTree.java:1028)
        at edu.uci.ics.hyracks.storage.am.btree.multithread.BTreeTestWorker.performOp(BTreeTestWorker.java:61)
        at edu.uci.ics.hyracks.storage.am.common.AbstractTreeIndexTestWorker.run(AbstractTreeIndexTestWorker.java:52)
edu.uci.ics.hyracks.storage.am.btree.exceptions.BTreeException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: Failed to pin page 
0:675 because all pages are pinned.
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:687)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:543)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:242)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.insert(BTree.java:259)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.access$1000(BTree.java:56)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.insert(BTree.java:1028)
        at edu.uci.ics.hyracks.storage.am.btree.multithread.BTreeTestWorker.performOp(BTreeTestWorker.java:61)
        at edu.uci.ics.hyracks.storage.am.common.AbstractTreeIndexTestWorker.run(AbstractTreeIndexTestWorker.java:52)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: Failed to 
pin page 0:675 because all pages are pinned.
        at edu.uci.ics.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:145)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:511)
        at edu.uci.ics.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:543)
        ... 7 more

The exception shows up just once, and I failed to reproduced it with the same 
code.

Original comment by jarod...@gmail.com on 10 May 2012 at 4:24

GoogleCodeExporter commented 9 years ago 
Thanks for the update. I'd say this is not a major concern because we 
deliberately run tests with a small buffercache to see how it behaves under 
heavy pin/unpin pressure. 

The exception you are seeing basically means that after some number of retries 
a pinner will bail if the buffercache can't find a victim page to evict.

Please let me know if this problem shows up again. I'll be watching it closely.

Btw, how many cores does your machine have?

Original comment by alexande...@gmail.com on 10 May 2012 at 7:53

GoogleCodeExporter commented 9 years ago 
Thanks Alex for the reply! So far I have not seen the same problem again. 

I have 4 cores for the test machine.

Original comment by jarod...@gmail.com on 10 May 2012 at 9:21