search

graphaware / neo4j-to-elasticsearch

GraphAware Framework Module for Integrating Neo4j with Elasticsearch
261 stars 57 forks source link

Delete node in neo4j but the index of it dont delete in elasticsearch #153

Closed whbjzzwjxq closed 3 years ago

whbjzzwjxq commented 5 years ago

My neo4j version is 3.5 and es version is 6.7

whbjzzwjxq commented 5 years ago

with many times restarting neo4j

ikwattro commented 5 years ago

List plugins versions + output of neo4j.log if available

whbjzzwjxq commented 5 years ago

the plugin version : graphaware-neo4j-to-elasticsearch-3.5.0.53.11.jar and the log when I delete the node

Actually I close the auto-creation of uuid for nodes

nohup: ignoring input 2019-05-21 11:02:57.994+0000 INFO ======== Neo4j 3.5.0 ======== 2019-05-21 11:02:58.004+0000 INFO Starting... 2019-05-21 11:03:00.075+0000 INFO GraphAware Runtime enabled, bootstrapping... 2019-05-21 11:03:00.098+0000 INFO Bootstrapping module with order 1, ID UIDM, using com.graphaware.module.uuid.UuidBootstrapper 2019-05-21 11:03:00.152+0000 INFO UIDM (re-)initialize until set to NEVER 2019-05-21 11:03:00.190+0000 INFO Node Inclusion Policy set to com.graphaware.common.policy.inclusion.composite.CompositeNodeInclusionPolicy@63fec893 2019-05-21 11:03:00.191+0000 INFO uuidProperty set to uuid 2019-05-21 11:03:00.191+0000 INFO uuidIndex set to uuidIndex 2019-05-21 11:03:00.191+0000 INFO uuidRelationshipIndex set to uuidRelIndex 2019-05-21 11:03:00.197+0000 INFO Registering module UIDM with GraphAware Runtime. 2019-05-21 11:03:00.197+0000 INFO Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper 2019-05-21 11:03:00.205+0000 INFO ElasticSearch index-prefix set to neo4j-index 2019-05-21 11:03:00.205+0000 INFO ElasticSearch key-property set to uuid 2019-05-21 11:03:00.205+0000 INFO ElasticSearch force-strings set to false 2019-05-21 11:03:00.205+0000 INFO ES (re-)initialize until set to 1231231231231230 2019-05-21 11:03:00.205+0000 INFO That's 1229672794651025 ms in the future 2019-05-21 11:03:00.205+0000 INFO Elasticsearch URI set to localhost 2019-05-21 11:03:00.205+0000 INFO Elasticsearch protocol set to http 2019-05-21 11:03:00.206+0000 INFO Elasticsearch port set to 9200 2019-05-21 11:03:00.206+0000 INFO Elasticsearch key property set to uuid 2019-05-21 11:03:00.206+0000 INFO Elasticsearch retry-on-error set to false 2019-05-21 11:03:00.206+0000 INFO Elasticsearch module queue capacity set to 10000 2019-05-21 11:03:00.206+0000 INFO Elasticsearch module reindex batch size set to 1000 2019-05-21 11:03:00.206+0000 INFO Elasticsearch bulk execution set to true 2019-05-21 11:03:00.212+0000 INFO Using mapping file at path /home/yu/neo4j-community-3.5.0/conf/mapping.json 2019-05-21 11:03:00.481+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.JsonFileMapping 2019-05-21 11:03:00.498+0000 INFO Registering module ES with GraphAware Runtime. 2019-05-21 11:03:00.499+0000 INFO GraphAware Runtime bootstrapped, starting the Runtime... 2019-05-21 11:03:00.648+0000 INFO Starting GraphAware... 2019-05-21 11:03:00.652+0000 INFO Loading module metadata... 2019-05-21 11:03:00.653+0000 INFO Loading metadata for module UIDM 2019-05-21 11:03:00.824+0000 INFO Module UIDM seems to have been registered before, metadata loaded successfully. 2019-05-21 11:03:00.824+0000 INFO Module UIDM seems to have changed configuration since last run, will try to re-initialize... 2019-05-21 11:03:00.825+0000 INFO InitializeUntil set to 0 and it is 1558436580825. Will NOT re-initialize. 2019-05-21 11:03:00.842+0000 INFO Loading metadata for module ES 2019-05-21 11:03:01.023+0000 ERROR Could not deserialize metadata for module ID ES 2019-05-21 11:03:01.025+0000 INFO Module ES seems to have corrupted metadata. 2019-05-21 11:03:01.025+0000 INFO Module ES seems to have corrupted metadata, will try to re-initialize... 2019-05-21 11:03:01.026+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558436581026. Will re-initialize. 2019-05-21 11:03:01.035+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558436581035. Will re-index the entire database... 2019-05-21 11:03:01.035+0000 INFO Creating fresh metadata for module ES. 2019-05-21 11:03:01.036+0000 INFO Module ES has not changed configuration since last run, already initialized. 2019-05-21 11:03:01.096+0000 INFO Module metadata loaded. 2019-05-21 11:03:01.097+0000 INFO Starting transaction-driven modules... 2019-05-21 11:03:01.097+0000 INFO Starting Elasticsearch Writer... 2019-05-21 11:03:01.113+0000 INFO Creating Jest Client... SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 2019-05-21 11:03:01.321+0000 INFO Created Jest Client. 2019-05-21 11:03:01.601+0000 INFO Started Elasticsearch Writer. 2019-05-21 11:03:01.603+0000 INFO Re-indexing nodes... 2019-05-21 11:03:02.485+0000 INFO Bulk operation succeeded 2019-05-21 11:03:02.485+0000 INFO Skipping relationships indexation. 2019-05-21 11:03:02.485+0000 INFO Finished re-indexing database. 2019-05-21 11:03:02.485+0000 INFO Transaction-driven modules started. 2019-05-21 11:03:02.485+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks. 2019-05-21 11:03:02.486+0000 INFO GraphAware started. 2019-05-21 11:03:02.494+0000 INFO GraphAware Runtime automatically started. 2019-05-21 11:03:06.642+0000 INFO Bolt enabled on 0.0.0.0:7687. 2019-05-21 11:03:08.469+0000 INFO Started. 2019-05-21 11:03:08.700+0000 INFO started 2019-05-21 11:03:08.702+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware] 2019-05-21 11:03:08.889+0000 INFO Google Analytics enabled 2019-05-21 11:03:08.902+0000 INFO Mounting GraphAware Framework at /graphaware 2019-05-21 11:03:08.907+0000 INFO Will try to scan the following packages: {com..graphaware.,org..graphaware.,net..graphaware.} 2019-05-21 11:03:11.572+0000 INFO Remote interface available at http://39.96.10.154:7474/ 2019-05-21 11:05:52.333+0000 INFO Neo4j Server shutdown initiated by request 2019-05-21 11:05:52.364+0000 INFO stopped 2019-05-21 11:05:52.364+0000 INFO Stopping... 2019-05-21 11:05:52.516+0000 INFO Shutting down GraphAware Runtime... 2019-05-21 11:05:52.517+0000 INFO Shutting down module UIDM 2019-05-21 11:05:52.517+0000 INFO Shutting down module ES 2019-05-21 11:05:52.517+0000 INFO Stopping Elasticsearch Writer... 2019-05-21 11:05:52.518+0000 INFO Shutting down Jest Client... 2019-05-21 11:05:52.518+0000 INFO Shut down Jest Client. 2019-05-21 11:05:52.518+0000 INFO Stopped Elasticsearch Writer. 2019-05-21 11:05:52.518+0000 INFO Terminating task scheduler... 2019-05-21 11:05:52.518+0000 INFO Task scheduler terminated successfully. 2019-05-21 11:05:52.518+0000 INFO GraphAware Runtime shut down. 2019-05-21 11:05:52.528+0000 INFO Stopped. nohup: ignoring input 2019-05-21 11:05:54.755+0000 INFO ======== Neo4j 3.5.0 ======== 2019-05-21 11:05:54.764+0000 INFO Starting... 2019-05-21 11:05:56.965+0000 INFO GraphAware Runtime enabled, bootstrapping... 2019-05-21 11:05:56.990+0000 INFO Bootstrapping module with order 1, ID UIDM, using com.graphaware.module.uuid.UuidBootstrapper 2019-05-21 11:05:57.049+0000 INFO UIDM (re-)initialize until set to NEVER 2019-05-21 11:05:57.090+0000 INFO Node Inclusion Policy set to com.graphaware.common.policy.inclusion.composite.CompositeNodeInclusionPolicy@63fec893 2019-05-21 11:05:57.090+0000 INFO uuidProperty set to uuid 2019-05-21 11:05:57.090+0000 INFO uuidIndex set to uuidIndex 2019-05-21 11:05:57.090+0000 INFO uuidRelationshipIndex set to uuidRelIndex 2019-05-21 11:05:57.097+0000 INFO Registering module UIDM with GraphAware Runtime. 2019-05-21 11:05:57.098+0000 INFO Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper 2019-05-21 11:05:57.105+0000 INFO ElasticSearch index-prefix set to neo4j-index 2019-05-21 11:05:57.106+0000 INFO ElasticSearch key-property set to uuid 2019-05-21 11:05:57.106+0000 INFO ElasticSearch force-strings set to false 2019-05-21 11:05:57.106+0000 INFO ES (re-)initialize until set to 1231231231231230 2019-05-21 11:05:57.106+0000 INFO That's 1229672794474124 ms in the future 2019-05-21 11:05:57.106+0000 INFO Elasticsearch URI set to localhost 2019-05-21 11:05:57.106+0000 INFO Elasticsearch protocol set to http 2019-05-21 11:05:57.106+0000 INFO Elasticsearch port set to 9200 2019-05-21 11:05:57.106+0000 INFO Elasticsearch key property set to uuid 2019-05-21 11:05:57.107+0000 INFO Elasticsearch retry-on-error set to false 2019-05-21 11:05:57.107+0000 INFO Elasticsearch module queue capacity set to 10000 2019-05-21 11:05:57.107+0000 INFO Elasticsearch module reindex batch size set to 1000 2019-05-21 11:05:57.107+0000 INFO Elasticsearch bulk execution set to true 2019-05-21 11:05:57.113+0000 INFO Using mapping file at path /home/yu/neo4j-community-3.5.0/conf/mapping.json 2019-05-21 11:05:57.379+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.JsonFileMapping 2019-05-21 11:05:57.426+0000 INFO Registering module ES with GraphAware Runtime. 2019-05-21 11:05:57.430+0000 INFO GraphAware Runtime bootstrapped, starting the Runtime... 2019-05-21 11:05:57.630+0000 INFO Starting GraphAware... 2019-05-21 11:05:57.639+0000 INFO Loading module metadata... 2019-05-21 11:05:57.639+0000 INFO Loading metadata for module UIDM 2019-05-21 11:05:57.779+0000 INFO Module UIDM seems to have been registered before, metadata loaded successfully. 2019-05-21 11:05:57.780+0000 INFO Module UIDM seems to have changed configuration since last run, will try to re-initialize... 2019-05-21 11:05:57.781+0000 INFO InitializeUntil set to 0 and it is 1558436757781. Will NOT re-initialize. 2019-05-21 11:05:57.792+0000 INFO Loading metadata for module ES 2019-05-21 11:05:58.016+0000 ERROR Could not deserialize metadata for module ID ES 2019-05-21 11:05:58.017+0000 INFO Module ES seems to have corrupted metadata. 2019-05-21 11:05:58.017+0000 INFO Module ES seems to have corrupted metadata, will try to re-initialize... 2019-05-21 11:05:58.019+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558436758019. Will re-initialize. 2019-05-21 11:05:58.019+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558436758019. Will re-index the entire database... 2019-05-21 11:05:58.019+0000 INFO Creating fresh metadata for module ES. 2019-05-21 11:05:58.019+0000 INFO Module ES has not changed configuration since last run, already initialized. 2019-05-21 11:05:58.079+0000 INFO Module metadata loaded. 2019-05-21 11:05:58.080+0000 INFO Starting transaction-driven modules... 2019-05-21 11:05:58.081+0000 INFO Starting Elasticsearch Writer... 2019-05-21 11:05:58.105+0000 INFO Creating Jest Client... SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 2019-05-21 11:05:58.482+0000 INFO Created Jest Client. 2019-05-21 11:05:58.700+0000 INFO Started Elasticsearch Writer. 2019-05-21 11:05:58.703+0000 INFO Re-indexing nodes... 2019-05-21 11:05:59.450+0000 INFO Bulk operation succeeded 2019-05-21 11:05:59.450+0000 INFO Skipping relationships indexation. 2019-05-21 11:05:59.450+0000 INFO Finished re-indexing database. 2019-05-21 11:05:59.450+0000 INFO Transaction-driven modules started. 2019-05-21 11:05:59.450+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks. 2019-05-21 11:05:59.466+0000 INFO GraphAware started. 2019-05-21 11:05:59.466+0000 INFO GraphAware Runtime automatically started. 2019-05-21 11:06:04.006+0000 INFO Bolt enabled on 0.0.0.0:7687. 2019-05-21 11:06:05.725+0000 INFO Started. 2019-05-21 11:06:06.090+0000 INFO started 2019-05-21 11:06:06.097+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware] 2019-05-21 11:06:06.333+0000 INFO Google Analytics enabled 2019-05-21 11:06:06.343+0000 INFO Mounting GraphAware Framework at /graphaware 2019-05-21 11:06:06.348+0000 INFO Will try to scan the following packages: {com..graphaware.,org..graphaware.,net..graphaware.} 2019-05-21 11:06:09.202+0000 INFO Remote interface available at http://39.96.10.154:7474/ 2019-05-21 11:18:24.388+0000 INFO Neo4j Server shutdown initiated by request 2019-05-21 11:18:24.407+0000 INFO stopped 2019-05-21 11:18:24.407+0000 INFO Stopping... 2019-05-21 11:18:24.493+0000 INFO Shutting down GraphAware Runtime... 2019-05-21 11:18:24.493+0000 INFO Shutting down module UIDM 2019-05-21 11:18:24.493+0000 INFO Shutting down module ES 2019-05-21 11:18:24.493+0000 INFO Stopping Elasticsearch Writer... 2019-05-21 11:18:24.494+0000 INFO Shutting down Jest Client... 2019-05-21 11:18:24.494+0000 INFO Shut down Jest Client. 2019-05-21 11:18:24.494+0000 INFO Stopped Elasticsearch Writer. 2019-05-21 11:18:24.494+0000 INFO Terminating task scheduler... 2019-05-21 11:18:24.494+0000 INFO Task scheduler terminated successfully. 2019-05-21 11:18:24.495+0000 INFO GraphAware Runtime shut down. 2019-05-21 11:18:24.503+0000 INFO Stopped. nohup: ignoring input 2019-05-21 11:18:26.801+0000 INFO ======== Neo4j 3.5.0 ======== 2019-05-21 11:18:26.811+0000 INFO Starting... 2019-05-21 11:18:28.849+0000 INFO GraphAware Runtime enabled, bootstrapping... 2019-05-21 11:18:28.875+0000 INFO Bootstrapping module with order 1, ID UIDM, using com.graphaware.module.uuid.UuidBootstrapper 2019-05-21 11:18:28.935+0000 INFO UIDM (re-)initialize until set to NEVER 2019-05-21 11:18:28.977+0000 INFO Node Inclusion Policy set to com.graphaware.common.policy.inclusion.composite.CompositeNodeInclusionPolicy@63fec893 2019-05-21 11:18:28.977+0000 INFO uuidProperty set to uuid 2019-05-21 11:18:28.978+0000 INFO uuidIndex set to uuidIndex 2019-05-21 11:18:28.978+0000 INFO uuidRelationshipIndex set to uuidRelIndex 2019-05-21 11:18:28.985+0000 INFO Registering module UIDM with GraphAware Runtime. 2019-05-21 11:18:28.986+0000 INFO Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper 2019-05-21 11:18:28.994+0000 INFO ElasticSearch index-prefix set to neo4j-index 2019-05-21 11:18:28.996+0000 INFO ElasticSearch key-property set to uuid 2019-05-21 11:18:28.996+0000 INFO ElasticSearch force-strings set to false 2019-05-21 11:18:28.996+0000 INFO ES (re-)initialize until set to 1231231231231230 2019-05-21 11:18:28.996+0000 INFO That's 1229672793722234 ms in the future 2019-05-21 11:18:28.996+0000 INFO Elasticsearch URI set to localhost 2019-05-21 11:18:28.997+0000 INFO Elasticsearch protocol set to http 2019-05-21 11:18:28.997+0000 INFO Elasticsearch port set to 9200 2019-05-21 11:18:28.997+0000 INFO Elasticsearch key property set to uuid 2019-05-21 11:18:28.998+0000 INFO Elasticsearch retry-on-error set to false 2019-05-21 11:18:28.998+0000 INFO Elasticsearch module queue capacity set to 10000 2019-05-21 11:18:29.001+0000 INFO Elasticsearch module reindex batch size set to 1000 2019-05-21 11:18:29.001+0000 INFO Elasticsearch bulk execution set to true 2019-05-21 11:18:29.008+0000 INFO Using mapping file at path /home/yu/neo4j-community-3.5.0/conf/mapping.json 2019-05-21 11:18:29.279+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.JsonFileMapping 2019-05-21 11:18:29.334+0000 INFO Registering module ES with GraphAware Runtime. 2019-05-21 11:18:29.346+0000 INFO GraphAware Runtime bootstrapped, starting the Runtime... 2019-05-21 11:18:29.522+0000 INFO Starting GraphAware... 2019-05-21 11:18:29.530+0000 INFO Loading module metadata... 2019-05-21 11:18:29.530+0000 INFO Loading metadata for module UIDM 2019-05-21 11:18:29.684+0000 INFO Module UIDM seems to have been registered before, metadata loaded successfully. 2019-05-21 11:18:29.684+0000 INFO Module UIDM seems to have changed configuration since last run, will try to re-initialize... 2019-05-21 11:18:29.684+0000 INFO InitializeUntil set to 0 and it is 1558437509684. Will NOT re-initialize. 2019-05-21 11:18:29.699+0000 INFO Loading metadata for module ES 2019-05-21 11:18:29.863+0000 ERROR Could not deserialize metadata for module ID ES 2019-05-21 11:18:29.863+0000 INFO Module ES seems to have corrupted metadata. 2019-05-21 11:18:29.863+0000 INFO Module ES seems to have corrupted metadata, will try to re-initialize... 2019-05-21 11:18:29.863+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558437509863. Will re-initialize. 2019-05-21 11:18:29.863+0000 INFO InitializeUntil set to 1231231231231230 and it is 1558437509863. Will re-index the entire database... 2019-05-21 11:18:29.863+0000 INFO Creating fresh metadata for module ES. 2019-05-21 11:18:29.863+0000 INFO Module ES has not changed configuration since last run, already initialized. 2019-05-21 11:18:29.952+0000 INFO Module metadata loaded. 2019-05-21 11:18:29.953+0000 INFO Starting transaction-driven modules... 2019-05-21 11:18:29.953+0000 INFO Starting Elasticsearch Writer... 2019-05-21 11:18:29.981+0000 INFO Creating Jest Client... SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 2019-05-21 11:18:30.152+0000 INFO Created Jest Client. 2019-05-21 11:18:30.292+0000 INFO Started Elasticsearch Writer. 2019-05-21 11:18:30.298+0000 INFO Re-indexing nodes... 2019-05-21 11:18:31.089+0000 INFO Bulk operation succeeded 2019-05-21 11:18:31.089+0000 INFO Skipping relationships indexation. 2019-05-21 11:18:31.090+0000 INFO Finished re-indexing database. 2019-05-21 11:18:31.090+0000 INFO Transaction-driven modules started. 2019-05-21 11:18:31.090+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks. 2019-05-21 11:18:31.090+0000 INFO GraphAware started. 2019-05-21 11:18:31.090+0000 INFO GraphAware Runtime automatically started. 2019-05-21 11:18:35.453+0000 INFO Bolt enabled on 0.0.0.0:7687. 2019-05-21 11:18:37.511+0000 INFO Started. 2019-05-21 11:18:37.872+0000 INFO started 2019-05-21 11:18:37.874+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware] 2019-05-21 11:18:38.308+0000 INFO Google Analytics enabled 2019-05-21 11:18:38.327+0000 INFO Mounting GraphAware Framework at /graphaware 2019-05-21 11:18:38.335+0000 INFO Will try to scan the following packages: {com..graphaware.,org..graphaware.,net..graphaware.} 2019-05-21 11:18:41.146+0000 INFO Remote interface available at http://39.96.10.154:7474/

whbjzzwjxq commented 5 years ago

I search the keyword in es and the response is Repetitive

{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 6.8550067, "hits" : [ { "_index" : "nodes-strnode", "_type" : "strnodes", "_id" : "9ccfba0d-1d71-11e9-8395-00163e2ebf28", "_score" : 6.8550067, "_source" : { "name_zh" : "拉菲尔·莫内欧", "name" : " Rafael Moneo ", "name_en" : null } }, { "_index" : "nodes-strnode", "_type" : "strnodes", "_id" : "42437790-2049-11e9-8395-00163e2ebf28", "_score" : 6.38458, "_source" : { "name_zh" : null, "name" : "Rafael Moneo", "name_en" : null } } ] } }

but the node whose uuid is "9ccfba0d-1d71-11e9-8395-00163e2ebf28" was deleted in 2019-05-21