update data between neo4j and elasticsearch !

[sofiane01234]

Hi sir @ikwattro ,

I need to integrate a Neo4j database with Elasticsearch to do some analytics and monitoring.

In my neo4j database, I have many relationship types that I want to duplicate to Elasticsearch : one-to-one; one-to-many; many-to-many. But, I didn't want to use any of the mapping strategies availible in ES : Nested Object ; Parent-Child... Some of the reasons I don't want to use those predefined strategies are :

• I want to be able to manage the updates in relationships easily (automate it or semi-automate it) ;
• I don't want to limit myself in term of data modeling and mapping since I didn't decide yet If i will be using one or more that one index .

So I've implemented a new strategy which I'll introduce in the following example:

In my neo4j database, I have the following relationship between a node having the label 'Member' and an other one having the label 'Account', which is one-to-one relationship :

(m:Member)-[r:HAS_ACCOUNT]->(a:Account)

Instead of using Nested object strategy, I'm indexing only Member nodes :

• each Member node is indexed in one ES document;
• each Member document will contain attributes from the Account node that is linked to the Member node. Those attribute are retreived using a Cypher query.

This is the mapping.json file that I'm using for this strategy. ​

​ For the update case, when updating an Account node in Neo4j, its member document must be updated accordingly. To acheive that, I've implemented a fictive field in Member node , in neo4j (which is also indexed in ES) called "updated". This field will be updated in Neo4j, every time when one of the attributes of an Account node are updated (in this example, it's "email" field). This will force the reindexation of its Member document in ES.

My question for you is : Do you approve this strategy ? Do you think that it can alter the performances of ES, Neo4j or Grapheaware ?

Here are the versions of the tools I'm using : graphaware-framework version : graphaware-server-community-all-3.4.0.52 neo4j-to-elasticsearch version : graphaware-neo4j-to-elasticsearch-3.4.0.52.9-SNAPSHOT neo4j-uuid version : graphaware-uuid-3.4.0.52.17 elasticsearch version : elasticsearch-6.2.4 neo4j version : neo4j-community-3.4.0 Thanks in advance

[ikwattro]

I don't see any issue using nested fields, especially as it can contains more information, is more flexible in case multiple account per users can be created. Also, as it is a general question, please gear it towards StackOverflow with the corresponding tags. Thank you.