graphaware / neo4j-to-elasticsearch

GraphAware Framework Module for Integrating Neo4j with Elasticsearch
261 stars 57 forks source link

using Cypher queries in Relationship mapping #129

Closed Hadjerkhd closed 6 years ago

Hadjerkhd commented 6 years ago

Hello @ikwattro, I have the following relations that I want to index into ES, by simulating a parent-child join :

Member-[:ADD_COMMENT]->Comment Member-[:VOTES]->Comment

While Comment is the child and Member is the parent in both cases , and since ES allows only one parent per child, I'm indexing the relations :ADD_COMMENT and :VOTES. I wanted to add some properties to the indexed relations using Cypher queries , to do so, I've used the following mapping file :

{
    "defaults": {
        "key_property": "uuid",
        "nodes_index": "neo4j-index-node",
        "relationships_index": "neo4j-index-relationship",
        "include_remaining_properties": false
    },
    "node_mappings": [

          {
            "condition": "hasLabel('Member')",
            "type": "doc",
            "properties": {

                "lname": "query('MATCH (n:Member) WHERE id(n)={id} return n.lname+ \" \" + n.lname as value')",                                
                "fname": "query('MATCH (n:Member) WHERE id(n)={id} return n.lname+ \" \" + n.fname as value')",                
                "email": "query('MATCH (n:Member) WHERE id(n)={id} match (n)-[:HAS_ACCOUNT]->(a:Account) return  a.email as value')"

            }
        }

    ],
    "relationship_mappings": [

        {
            "condition": "isType('ADD_COMMENT')",
            "type": "doc", 
            "properties": {
                "com_uuid": "query ('match (re:Comment)<-[r:ADD_COMMENT]-(m:Member) where id(r) = {id} return re.uuid as value ')"

                }
        }, 
  {
            "condition": "isType('VOTES')",
            "type": "doc", 
            "properties": {
                "com_uuid": "query ('match (re:Comment)<-[r:VOTES]-(m:Member) where id(r) = {id} return re.uuid as value ')"

                }
        }

    ]
}

The nodes are well indexed , yet the relations are not and I got the following exception :

`2018-07-11 09:56:35.847+0000 ERROR Error while creating relationship: com.graphaware.module.es.mapping.expression.RelationshipExpressions@c5b9aa54 EL1004E: Method call: Method query(java.lang.String) cannot be found on com.graphaware.module.es.mapping.expression.RelationshipExpressions type
org.springframework.expression.spel.SpelEvaluationException: EL1004E: Method call: Method query(java.lang.String) cannot be found on com.graphaware.module.es.mapping.expression.RelationshipExpressions type
        at org.springframework.expression.spel.ast.MethodReference.findAccessorForMethod(MethodReference.java:212)
        at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:126)
        at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:86)
        at org.springframework.expression.spel.ast.SpelNodeImpl.getValue(SpelNodeImpl.java:121)
        at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:199)
        at com.graphaware.module.es.mapping.json.GraphDocumentMapper.getDocumentRepresentation(GraphDocumentMapper.java:190)
        at com.graphaware.module.es.mapping.json.GraphDocumentMapper.getDocumentRepresentation(GraphDocumentMapper.java:180)
        at com.graphaware.module.es.mapping.json.DocumentMappingRepresentation.createOrUpdateRelationship(DocumentMappingRepresentation.java:95)
        at com.graphaware.module.es.mapping.JsonFileMapping.createRelationship(JsonFileMapping.java:79)
        at com.graphaware.module.es.mapping.Mapping.getActions(Mapping.java:61)
        at com.graphaware.module.es.ElasticSearchWriter.processOperations(ElasticSearchWriter.java:116)
        at com.graphaware.module.es.ElasticSearchModule.reindexRelationships(ElasticSearchModule.java:224)
        at com.graphaware.module.es.ElasticSearchModule.lambda$reindex$0(ElasticSearchModule.java:153)
        at com.graphaware.module.es.ElasticSearchModule.reindex(ElasticSearchModule.java:166)
        at com.graphaware.module.es.ElasticSearchModule.start(ElasticSearchModule.java:94)
        at com.graphaware.runtime.manager.ProductionTxDrivenModuleManager.start(ProductionTxDrivenModuleManager.java:49)
        at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.startModules(BaseTxDrivenModuleManager.java:113)
        at com.graphaware.runtime.TxDrivenRuntime.startModules(TxDrivenRuntime.java:147)
        at com.graphaware.runtime.ProductionRuntime.startModules(ProductionRuntime.java:70)
        at com.graphaware.runtime.BaseGraphAwareRuntime.start(BaseGraphAwareRuntime.java:134)
        at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.lambda$start$9(RuntimeKernelExtension.java:117)
        at java.lang.Thread.run(Thread.java:745)`

From my understanding to this exception, cypher queries are not allowed in relationship mappings .. Is what I said correct ? if so , do you have any workaround ? or any plan to support this feature in the following version of the plugin ?

Thanks in advance !

ikwattro commented 6 years ago

@Hadjer13 Yes, I will add this possibility, the query feature is very recent, so it was added initially on nodes only for seeing the usage. But it will ship soon !

ikwattro commented 6 years ago

@Hadjer13 Done and released : https://products.graphaware.com/download/neo4j-to-elasticsearch/graphaware-neo4j-to-elasticsearch-3.4.0.52.10.jar

Askir commented 5 years ago

Is there any in depth documentation on how exactly this feature works? Where can I find how the id(r)={id} works for example?

I'm trying this:

{
      "condition": "hasLabel('Image')",
      "index": "node-image",
      "type": "image",
      "properties": {
        "url": "getProperty('url')",
        "tags": "query('MATCH (i:Image)-[:tagged_with]->(t:Tag) where id(i) = {id} RETURN collect(t.name) AS value')"
      }
    }

but tags always ends up with an empty array in elasticsearch after this create statement: CREATE (a:Image{url:'somerandomvalue'})-[:tagged_with]->(t:Tag{name:'arandomtag'})

Any help on how I could get that to work woudl be very appreciated.