search

gentics / mesh

Gentics Mesh - The open source headless CMS for developers
https://getmesh.io
Apache License 2.0
572 stars 115 forks source link

Revoking/Granting Permissions in parallel leads to conflict errors #382

Open philippguertler opened 6 years ago

philippguertler commented 6 years ago

Gentics Mesh Version, operating system, or hardware.

Operating System

Problem

When revoking/granting permissions in parallel on the same node for different roles, I got this error:

2018-03-19 13:08:43,694 [TVC Prod] ERROR [ElasticSearchProvider.java:372] - Updating object {840a08262c1243158a08262c12f3159f:default} to index failed. Duration 6[ms]
org.elasticsearch.index.engine.VersionConflictEngineException: [default][840a08262c1243158a08262c12f3159f]: version conflict, current [2], provided [1]
        at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:514) ~[tvc-server.jar:na]
        at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:454) ~[tvc-server.jar:na]
        at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:605) ~[tvc-server.jar:na]
        at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:236) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:66) ~[tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:657) ~[tvc-server.jar:na]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:287) ~[tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279) ~[tvc-server.jar:na]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) ~[tvc-server.jar:na]
        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:378) ~[tvc-server.jar:na]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [tvc-server.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_111-internal]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_111-internal]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111-internal]
2018-03-19 13:08:43,695 [TVC Prod] ERROR [SearchQueueBatchImpl.java:278] - Error while processing batch {null}
2018-03-19 13:08:43,702 [TVC Prod] ERROR [OrientDBDatabase.java:862] - Error handling transaction
org.elasticsearch.index.engine.VersionConflictEngineException: [default][840a08262c1243158a08262c12f3159f]: version conflict, current [2], provided [1]
        at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:514) ~[tvc-server.jar:na]
        at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:454) ~[tvc-server.jar:na]
        at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:605) ~[tvc-server.jar:na]
        at org.elasticsearch.index.engine.Engine$Index.execute(Engine.java:836) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:236) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157) ~[tvc-server.jar:na]
        at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:66) ~[tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:657) ~[tvc-server.jar:na]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:287) ~[tvc-server.jar:na]
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279) ~[tvc-server.jar:na]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) ~[tvc-server.jar:na]
        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:378) ~[tvc-server.jar:na]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[tvc-server.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_111-internal]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_111-internal]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111-internal]

Reproducer

Expected behaviour and actual behaviour

I expected all the roles to have permissions to the node. Instead I got the above conflict error message.

Additional Info

I did not expect a conflict error on permission changes. This probably happens because the read permission is saved in the Elasticsearch document. One solution would be to use scripted updates with the retry_on_conflict parameter.

Jotschi commented 5 years ago

I don't think this issue can happen in single node mode anymore. It may however be still occurring when running updates across a multi-master cluster.