search

graphaware / neo4j-to-elasticsearch

GraphAware Framework Module for Integrating Neo4j with Elasticsearch
261 stars 57 forks source link

indexing a neo4j relation between nodes as parent-child ES relationship #128

Closed Hadjerkhd closed 3 years ago

Hadjerkhd commented 6 years ago

Hello @ikwattro , I have the following one-to-many relation that I want to index as a parent-child relation.

Member-[:MAKE]->Vote

where Member is the parent and Vote is the child.

I'm tried to use this mapping file : 

{
 "defaults": {
        "key_property": "uuid",
        "nodes_index": "neo4j-index-node",
        "relationships_index": "neo4j-index-relationship",
        "include_remaining_properties": false
    },
    "node_mappings": [
        {
            "condition": "hasLabel('Vote')",
            "index":"test_index",
            "type": "doc",
            "properties": {
                "update_elastic": "getProperty('updated')",
                "TopicLabel": "getProperty('topic')", 

                "my_join_field": {
                    "name": "made_parent",
                    "parent": "query('match (m:Member)-[:ADD_VOTE]>(r:Vote) where id(r) = {id}  return m.uuid as value')"
                  }
            }
        } , 
   {
            "condition": "hasLabel('Member')",
            "index":"trustiser_test",
            "type": "doc",
            "properties": {

                "update_elastic": "getProperty('updated')",

                "lname": "query('MATCH (n:Member) WHERE id(n)={id} return n.lname+ \" \" + n.lname as value')",                                
                "fname": "query('MATCH (n:Member) WHERE id(n)={id} return n.lname+ \" \" + n.fname as value')",                
                "email": "query('MATCH (n:Member) WHERE id(n)={id} match (n)-[:HAS_ACCOUNT]->(a:Account) return  a.email as value')"    ,
                "my_join_field": "member"

            }
        } ] , 
"relationship_mappings": []
}

But I got the following exeption when staring neo4j server : 


2018-07-10 14:07:27.125+0000 ERROR [c.g.r.b.RuntimeKernelExtension] Unable to bootstrap module 2 Unable to read json mapping file
java.lang.RuntimeException: Unable to read json mapping file
        at com.graphaware.module.es.mapping.JsonFileMapping.configure(JsonFileMapping.java:66)
        at com.graphaware.module.es.ElasticSearchConfiguration.withMapping(ElasticSearchConfiguration.java:146)
        at com.graphaware.module.es.ElasticSearchModuleBootstrapper.doBootstrapModule(ElasticSearchModuleBootstrapper.java:116)
        at com.graphaware.module.es.ElasticSearchModuleBootstrapper.doBootstrapModule(ElasticSearchModuleBootstrapper.java:31)
        at com.graphaware.runtime.module.BaseRuntimeModuleBootstrapper.bootstrapModule(BaseRuntimeModuleBootstrapper.java:66)
        at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.registerModules(RuntimeKernelExtension.java:143)
        at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.start(RuntimeKernelExtension.java:113)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.kernel.extension.KernelExtensions.start(KernelExtensions.java:84)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:208)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:125)
        at org.neo4j.server.CommunityNeoServer.lambda$static$0(CommunityNeoServer.java:58)
        at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:88)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:212)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:111)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:79)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: (File); line: 17, column: 34] (through reference chain: com.graphaware.module.es.mapping.json.DocumentMappingRepresentation["node_mappings"]->java.util.ArrayList[0]->com.graphaware.module.es.mapping.json.GraphDocumentMapper["properties"]->java.util.LinkedHashMap["my_join_field"])
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
        at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1329)
        at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1138)
        at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1092)
        at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:59)
        at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:10)
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
        at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:136)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:287)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:285)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
        at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:136)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:287)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2890)
        at com.graphaware.module.es.mapping.JsonFileMapping.configure(JsonFileMapping.java:64)
        ... 21 more
2018-07-10 14:07:27.127+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime bootstrapped, starting the Runtime...
2018-07-10 14:07:30.268+0000 ERROR Unexpected error detected in bolt session '08d40cfffe60a144-00004b88-00000001-1a5cbc85abe3613a-9172ad73'. The client is unauthorized due to authentication failure.
org.neo4j.bolt.v1.runtime.BoltConnectionFatality: The client is unauthorized due to authentication failure.
        at org.neo4j.bolt.v1.runtime.BoltStateMachine.handleFailure(BoltStateMachine.java:740)
        at org.neo4j.bolt.v1.runtime.BoltStateMachine.handleFailure(BoltStateMachine.java:726)
        at org.neo4j.bolt.v1.runtime.BoltStateMachine.access$300(BoltStateMachine.java:62)
        at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$1.init(BoltStateMachine.java:434)
        at org.neo4j.bolt.v1.runtime.BoltStateMachine.init(BoltStateMachine.java:140)
        at org.neo4j.bolt.v1.messaging.BoltMessageRouter.lambda$onInit$0(BoltMessageRouter.java:70)
        at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:193)
        at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:143)
        at org.neo4j.bolt.runtime.ExecutorBoltScheduler.executeBatch(ExecutorBoltScheduler.java:163)
        at org.neo4j.bolt.runtime.ExecutorBoltScheduler.lambda$null$0(ExecutorBoltScheduler.java:145)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

the pointed line in the code [Source: (File); line: 17, column: 34] refers to this "my_join_field": {

I hope I can find an answer to my question here. Thanks in advance !

ikwattro commented 6 years ago

Hi @Hadjer13

You cannot pass custom string values in a property field like "my_join_field": "member" because those are evaluated. However I understand the use case, so we will add this possibility in the next release.

Also, this is not valid, and I don't think we will support nested syntax, best is to return a map from a query

"my_join_field": {
                    "name": "made_parent",
                    "parent": "query('match (m:Member)-[:ADD_VOTE]>(r:Vote) where id(r) = {id}  return m.uuid as value')"
                  }
Hadjerkhd commented 6 years ago

Thank you sir for your quick relpay , I think I'll go for the 'return map from a query' option to implement this , then change the mapping of the index in ES to parent-child join. And I hope this is will give the expected result.