search

tzaeschke / zoodb

ZooDB Object Database
Apache License 2.0
57 stars 9 forks source link

"Page type mismatch" after removing index #109

Closed tzaeschke closed 4 years ago

tzaeschke commented 6 years ago

A lengthy combination of repeatedly creating/removing an index an dropping all associated instances can result in a "page type mismatch error", see Test_092_xxx:

javax.jdo.JDOFatalInternalException: Page type mismatch, expected 4/OID_INDEX (tx=0) but got 5 (tx=10). PageId=7
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at org.zoodb.internal.util.ReflTools.newInstance(ReflTools.java:211)
    at org.zoodb.internal.util.DBLogger.newEx(DBLogger.java:111)
    at org.zoodb.internal.util.DBLogger.newEx(DBLogger.java:102)
    at org.zoodb.internal.util.DBLogger.newFatalInternal(DBLogger.java:173)
    at org.zoodb.internal.util.DBLogger.newFatalInternal(DBLogger.java:169)
    at org.zoodb.internal.server.StorageReader.readHeader(StorageReader.java:334)
    at org.zoodb.internal.server.StorageReader.seekPage(StorageReader.java:104)
    at org.zoodb.internal.server.StorageReader.seekPageForRead(StorageReader.java:80)
    at org.zoodb.internal.server.index.AbstractPagedIndex.readPage(AbstractPagedIndex.java:173)
    at org.zoodb.internal.server.index.AbstractIndexPage.readOrCreatePage(AbstractIndexPage.java:160)
    at org.zoodb.internal.server.index.LLIndexPage.locatePageForKey(LLIndexPage.java:195)
    at org.zoodb.internal.server.index.LLIndexPage.locatePageForKeyUnique(LLIndexPage.java:167)
    at org.zoodb.internal.server.index.PagedUniqueLongLong.findValue(PagedUniqueLongLong.java:117)
    at org.zoodb.internal.server.index.PagedOidIndex.findOid(PagedOidIndex.java:224)
    at org.zoodb.internal.server.DiskAccessOneFile.readObject(DiskAccessOneFile.java:393)
    at org.zoodb.internal.server.DiskAccessOneFile.readObject(DiskAccessOneFile.java:330)
    at org.zoodb.internal.server.index.SchemaIndex.readSchemaAll(SchemaIndex.java:532)
    at org.zoodb.internal.server.DiskAccessOneFile.readSchemaAll(DiskAccessOneFile.java:177)
    at org.zoodb.internal.model1p.Node1P.connect(Node1P.java:77)
    at org.zoodb.internal.Session.<init>(Session.java:111)
    at org.zoodb.jdo.impl.PersistenceManagerImpl.<init>(PersistenceManagerImpl.java:101)
    at org.zoodb.jdo.impl.PersistenceManagerFactoryImpl.getPersistenceManager(PersistenceManagerFactoryImpl.java:97)
    at org.zoodb.test.testutil.TestTools.openPM(TestTools.java:150)
    at org.zoodb.test.testutil.TestTools.openPM(TestTools.java:144)
    at org.zoodb.test.jdo.Test_092_IndexQueryBug.test(Test_092_IndexQueryBug.java:84)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
NestedThrowablesStackTrace:

java.lang.IllegalStateException: Some files were not closed: 1
    at org.zoodb.tools.ZooDebug.closeOpenFiles(ZooDebug.java:69)
    at org.zoodb.test.testutil.TestTools.closePM(TestTools.java:193)
    at org.zoodb.test.jdo.Test_092_IndexQueryBug.afterTest(Test_092_IndexQueryBug.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
tzaeschke commented 5 years ago

Scheduled for 0.5.2 / 0.6.0

dominik-kopczynski commented 4 years ago

Hi there, we are running exactly into this problem. So is there any solution, because here I can't see anyone? Or are we supposed to wait until version 0.6? If we have so, what is the expected release time?

dominik-kopczynski commented 4 years ago

Ahh, it was fixed from version 0.5.1 to 0.5.2, isn't it? We still use 0.5.1.

tzaeschke commented 4 years ago

Yes, this particular issue was fixed in 0.5.2. If you are removing indexes, this may very well be fixed by using 0.5.2. Let me know if it does not.

Btw, I am currently preparing a 0.6.0, it will mainly fix issues with multiple concurrent PersistentManagers: #114 and #103 .

dominik-kopczynski commented 4 years ago

Sorry, but unfortunately we have to reopen the issue again. After recompiling our tool with ZooDB 0.5.2 we still get the "Page type mismatch" error. How can we proceed, should I provide you the corrupted database file?

The exception stack trace is: javax.jdo.JDOFatalInternalException: Page type mismatch, expected 4/OID_INDEX (tx=0) but got 0 (tx=0). PageId=718102 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.zoodb.internal.util.ReflTools.newInstance(ReflTools.java:211) at org.zoodb.internal.util.DBLogger.newEx(DBLogger.java:108) at org.zoodb.internal.util.DBLogger.newEx(DBLogger.java:99) at org.zoodb.internal.util.DBLogger.newFatalInternal(DBLogger.java:162) at org.zoodb.internal.util.DBLogger.newFatalInternal(DBLogger.java:158) at org.zoodb.internal.server.StorageReader.readHeader(StorageReader.java:334) at org.zoodb.internal.server.StorageReader.seekPage(StorageReader.java:104) at org.zoodb.internal.server.StorageReader.seekPageForRead(StorageReader.java:80) at org.zoodb.internal.server.index.AbstractPagedIndex.readPage(AbstractPagedIndex.java:173) at org.zoodb.internal.server.index.AbstractPagedIndex.readRoot(AbstractPagedIndex.java:160) at org.zoodb.internal.server.index.PagedUniqueLongLong.(PagedUniqueLongLong.java:57) at org.zoodb.internal.server.index.PagedUniqueLongLong.(PagedUniqueLongLong.java:52) at org.zoodb.internal.server.index.IndexFactory.loadUniqueIndex(IndexFactory.java:105) at org.zoodb.internal.server.index.PagedOidIndex.(PagedOidIndex.java:190) at org.zoodb.internal.server.SessionManager.(SessionManager.java:148) at org.zoodb.internal.server.SessionFactory.getSession(SessionFactory.java:88) at org.zoodb.internal.model1p.Node1P.connect(Node1P.java:74) at org.zoodb.internal.Session.(Session.java:112) at org.zoodb.jdo.impl.PersistenceManagerImpl.(PersistenceManagerImpl.java:101) at org.zoodb.jdo.impl.PersistenceManagerFactoryImpl.getPersistenceManager(PersistenceManagerFactoryImpl.java:97) at org.zoodb.jdo.ZooJdoHelper.openDB(ZooJdoHelper.java:54) at org.zoodb.jdo.ZooJdoHelper.openOrCreateDB(ZooJdoHelper.java:81)

tzaeschke commented 4 years ago

Yes, the database file may be helpful. However, it would be easiest if I had a program that reproduces the problem, or does it only happen with this database file? Could you also describe how the database got into this state, e.g. were any special operations executed, concurrent sessions, ... .

Is this still compomics-utilities? Which branch?

tzaeschke commented 4 years ago

I created a new issue #122 to cover this problem.