search

orientechnologies / orientdb

OrientDB is the most versatile DBMS supporting Graph, Document, Reactive, Full-Text and Geospatial models in one Multi-Model product. OrientDB can run distributed (Multi-Master), supports SQL, ACID Transactions, Full-Text indexing and Reactive Queries.
https://orientdb.dev
Apache License 2.0
4.74k stars 870 forks source link

Indexes with ignoreNullValues cause errors on new nodes in distributed mode #3769

Closed tobiemh closed 9 years ago

tobiemh commented 9 years ago

This error prevents being able to use indexes with { ignoreNullValues: false } when in distributed mode.

Start 3 servers in distributed mode (core-1, core-2, core-3)

CREATE DATABASE remote:172.17.8.101/testdatabase root root plocal graph

Wait for database to sync across 3 nodes. Then...

CONNECT remote:172.17.8.101/testdatabase root root
CREATE CLASS Person extends V
CREATE PROPERTY Person.name STRING
CREATE INDEX Person.name NOTUNIQUE METADATA { ignoreNullValues: false }

Then start node 4 and wait for database to sync across to new node (core-4). Then...

CONNECT remote:172.17.8.104/testdatabase root root

CREATE VERTEX Person SET name = 'Tobie'
# Created vertex 'Person#28:0{name:Tobie} v1' in 0.077000 sec(s).
# This query works fine

CREATE VERTEX Person SET temp = true
# Created vertex 'Person#28:1{temp:true} v1' in 0.068000 sec(s).
# This query causes the error, probably because it does not have a 'name' field

However an error occurs on the new node, and the data is now out of sync on the servers.

Error on distributed transaction commit
com.orientechnologies.orient.core.index.sbtree.local.OSBTreeException: Null keys are not supported.
   at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.checkNullSupport(OSBTree.java:769)
   at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.get(OSBTree.java:185)
   at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.get(OSBTreeIndexEngine.java:226)
   at com.orientechnologies.orient.core.index.OIndexMultiValues.putInSnapshot(OIndexMultiValues.java:162)
   at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1025)
   at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:690)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:98)
   at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:922)
   at com.orientechnologies.orient.server.distributed.ODistributedStorage.commit(ODistributedStorage.java:802)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:483)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:147)
   at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2376)
   at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2346)
   at com.orientechnologies.orient.server.distributed.task.OTxTask.execute(OTxTask.java:111)
   at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.executeOnLocalNode(OHazelcastPlugin.java:757)
   at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.onMessage(ODistributedWorker.java:300)
   at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.run(ODistributedWorker.java:121)
[core-4] error on executing request 65 (tx) on local node:
com.orientechnologies.orient.core.index.sbtree.local.OSBTreeException: Null keys are not supported.
   at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.checkNullSupport(OSBTree.java:769)
   at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.get(OSBTree.java:185)
   at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.get(OSBTreeIndexEngine.java:226)
   at com.orientechnologies.orient.core.index.OIndexMultiValues.putInSnapshot(OIndexMultiValues.java:162)
   at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1025)
   at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:690)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:98)
   at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:922)
   at com.orientechnologies.orient.server.distributed.ODistributedStorage.commit(ODistributedStorage.java:802)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:483)
   at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:147)
   at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2376)
   at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2346)
   at com.orientechnologies.orient.server.distributed.task.OTxTask.execute(OTxTask.java:111)
   at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.executeOnLocalNode(OHazelcastPlugin.java:757)
   at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.onMessage(ODistributedWorker.java:300)
   at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.run(ODistributedWorker.java:121)

2015-03-17 02:42:48:468 WARNING [core-4] detected 1 conflicts, but the quorum (2) has been reached. Fixing remote records. Request (id=65 from=core-4 task=tx userName=) [ODistributedResponseManager]

NOTE:

OrientDB version: 2.0.5

tobiemh commented 9 years ago

This still exists in OrientDB Version 2.0.6 and 2.0.7

The bug removes the possibility of using indexes with ignoreNullValues in a distributed setup. The bug prevents being able to add a database on 1 node, and then start up other nodes in distributed setup.

Rebuilding ALL of the indexes on ALL of the servers does not fix the issue.

If you try to create a vertex/edge which belong to a class which has an ignoreNullValues index then the following error will appear...

orient|com.orientechnologies.orient.core.index.sbtree.local.OSBTreeException: Null keys are not supported.
orient| at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.checkNullSupport(OSBTree.java:769)
orient| at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.get(OSBTree.java:185)
orient| at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.get(OSBTreeIndexEngine.java:226)
orient| at com.orientechnologies.orient.core.index.OIndexMultiValues.putInSnapshot(OIndexMultiValues.java:162)
orient| at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1025)
orient| at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:690)
orient| at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:98)
orient| at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:922)
orient| at com.orientechnologies.orient.server.distributed.ODistributedStorage.commit(ODistributedStorage.java:811)
orient| at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:483)
orient| at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:147)
orient| at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2406)
orient| at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2376)
orient| at com.orientechnologies.orient.server.distributed.task.OTxTask.execute(OTxTask.java:113)
orient| at com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin.executeOnLocalNode(OHazelcastPlugin.java:772)
orient| at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.onMessage(ODistributedWorker.java:300)
orient| at com.orientechnologies.orient.server.hazelcast.ODistributedWorker.run(ODistributedWorker.java:121)
tobiemh commented 9 years ago

Anyone have any idea how to get round this?

tobiemh commented 9 years ago

Hi @lvca - any possibility of this being marked as a bug?

tobiemh commented 9 years ago

Is there any latest info on this issue?

lvca commented 9 years ago

I created this test case and everything works: https://github.com/orientechnologies/orientdb/commit/f97cccb32860eedf8cda7b54983761290488a305. I don't know if this has been fixed in 2.1-SNAPSHOT (develop branch). Could you please try with 2.1-SNAPSHOT?

prjhub commented 9 years ago

The issue has been closed because no response has been received after a while. In case the issue is still valid, please reopen it or write a comment that explains why it's still valid and a way to reproduce it. Thank you.

cshingle commented 8 years ago

I ran into this exact issue in version 2.1.15. I have the following schema. 

{
   "@type":"d",
   "@rid":"#15:147",
   "@version":1,
   "@class":"MToken",
   "user":"#12:3",
   "expiration":1460820192,
   "lastUsed":1460776992,
   "description":null,
   "token":"3_26j8C0fvxyiyJg2-b_nie1ynSRA-1QldkEb2cKGmA",<-- unique non-null Index on this field
   "created":1460776992,
}

{
   "@type":"d",
   "@rid":"#12:3",
   "@version":160,
   "@class":"MUser",
   "email":"sample@sample.com", <-- unique non-null Index on this field
   "firstName":"Sample",
   "lastName":"User",
   "enabled":true,
   "properties":{
   },
   "created":1460614838,
   "loginDate":1460776984,
   "tokens":[
     <null>, <-- reference to deleted record
     ...
     <null>, <-- reference to deleted record
      "#15:147"
   ],
}

My Token and user objects have a circular reference that is enforced by java hooks. My application works perfectly when not running in distributed mode. When I enable distributed mode I got the same exception as @tobiemh. I was also getting zero records returned when I would query my Token.token index, or run the following query. SELECT FROM MToken where token = '3_26j8C0fvxyiyJg2-b_nie1ynSRA-1QldkEb2cKGmA' What I eventually figured out is that some of my User objects had references to deleted tokens in the tokens embedded link set due to deleted records that escaped my AFTER_DELETE hook. Once I removed these null links with the following command, distributed mode started working correctly. update MUser remove tokens = null The rebuild index functionality seems to be attempting to index these null references when in distributed mode.