search

orientechnologies / orientdb

OrientDB is the most versatile DBMS supporting Graph, Document, Reactive, Full-Text and Geospatial models in one Multi-Model product. OrientDB can run distributed (Multi-Master), supports SQL, ACID Transactions, Full-Text indexing and Reactive Queries.
https://orientdb.dev
Apache License 2.0
4.74k stars 870 forks source link

com.orientechnologies.orient.core.index.OIndexException: Error during insertion of key in index #6219

Closed satyasrc closed 8 years ago

satyasrc commented 8 years ago

Expected behavior and actual behavior

OrientDB version used: 2.1.10 I am uploading full dbpedia dataset into orientDB. I have created a schema in which i have some properties defined on my custom vertex and edge classes. There are some Hash indices also defined on some of those properties. I am parsing the dbpedia graph from top to bottom in breadth-first fashion from Jena TDB and inserting into OrientDB. But i am getting this OIndexException after insertion of 27151 nodes and 30605 edges, though there was considerable amount of system-memory left. All indices used are of type HASH_INDEX (UNIQUE, NOT_UNIQUE, and FULL_TEXT). The full stack trace is given below: Exception in thread "main" com.orientechnologies.orient.core.index.OIndexException: Error during insertion of key in index at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:162) at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:52) at com.orientechnologies.orient.core.index.OIndexAbstract.putInSnapshot(OIndexAbstract.java:965) at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1151) at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:762) at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:101) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1211) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:606) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:156) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2689) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2658) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.commit(ONetworkProtocolBinary.java:1225) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:404) at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:223) at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77) Caused by: com.orientechnologies.orient.core.exception.OStorageException at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:381) at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.put(OHashTableIndexEngine.java:153) at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:155) ... 14 more Caused by: java.lang.IllegalStateException: Extendible hashing tree in corrupted state. at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.getBucket(OLocalHashTable.java:1848) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1315) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1413) at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:373) ... 16 more

Running Mode

This is an urgent issue for us as we are trying to deploy OrientDB in our production environment. Any help is much appreciated. Thanks.

andrii0lomakin commented 8 years ago

Hi, We have already fixed a similar issue in the latest version. Could you try on 2.1.18 version and send us feedback whether issue fixed.

satyasrc commented 8 years ago

Ok, Let me try that. Thanks.

satyasrc commented 8 years ago

I am trying the same with version 2.2.0. However, i am not able to see any of the already inserted data while insertion code is running. The graph is showing as empty while previously, when using v2.1.10 i was able to see the already inserted data even while the insertion code (the same code) was running on the db. This is crucial for us, since this code would be running for days on the dbpedia & other linked datasets and we need to be able to see the already inserted data. Any thoughts on why this is happening!! Is this a bug ? Remember, i am using remote mode for precisely the same reason, that is, so that i am able to see already inserted data while the graph upload is still happening. Awaiting you reply. Thanks.

andrii0lomakin commented 8 years ago

@srthecodexpress could you provide me code which you use ?

satyasrc commented 8 years ago

@laa unfortunately, I can't. The code uses our proprietary libraries. But i can tell you that insertion is happening as OrientDB is printing out the info on the console of each inserted node and edge. And, it is also printing out that the particular node/edge already exists when i am backtracking to insert next DBpedia subtree, so definitely it does exist in OrientDB. But i can't see it in OrientDB studio!! The DB is showing as empty there!! I should point out that i am using OrientGraphTx mode with AutoStartTx set to false, as i am manully starting and committing transactions before and after each node/edge, property insert.

andrii0lomakin commented 8 years ago

@maggiolo00 could you check that ?

luigidellaquila commented 8 years ago

Hi @srthecodexpress

Do you think you can share the DB (zipped db folder)? Could you also try to access the db from OrientDB Console (CLI) and see if you can query it?

Thanks

Luigi

satyasrc commented 8 years ago

Hi @laa @luigidellaquila That problem is fixed. I had to set the setAutoStartTx property to false at the time of graph or factory creation itself whereas i was doing it after the graph creation in tx mode due to which a transaction was left open. This caused the other transaction commits to not show up in the db.

Thanks guys for your time and patience. :+1: