jadell / neo4jphp

PHP wrapper of the Neo4j REST interface
Other
532 stars 137 forks source link

Cypher query for batch #142

Closed ArkeologeN closed 10 years ago

ArkeologeN commented 10 years ago

Hi,

I didn't found any support or doc that supports this wrapper to make cypher batch queries. I already have an implementation of making 100s of cypher batch and I want to know if this wrapper supports this or planning to have?

Thanks,

jadell commented 10 years ago

WIll Cypher transactions work for your use case? https://github.com/jadell/neo4jphp/wiki/Cypher-Transactions

ArkeologeN commented 10 years ago

I'm not sure about it, seems something relative. Does Cypher transaction put all statements to neo4j for parallel batch processing?

jadell commented 10 years ago

No, but neither does the batch functionality. They both guarantee transactionality (all statements succeed or all are rolled-back) but neither of them does parallel processing.

ArkeologeN commented 10 years ago

They may not work, this process will be blocking and we're making 100s or 1000s of cypher queries, then it not react as it has to for batch.

Neo4j Batch REST API allows to make cypher like this: Cypher syntax over Batch

This is how it looks like in javascript:

{
     method: "POST",
     to: "/cypher",
     body: {
         query: " START p=node({profileId}) " +
         "MERGE (s:Skill {name:'" + skill.skill.name + "'}) ON CREATE SET s.name = {skill} ON MATCH SET s.name = {skill} " +
         " CREATE UNIQUE (p)-[:HAS_SKILL]->(s)" +
         " RETURN s,p",
         params: {
              skill: skill.skill.name,
              profileId: profileId
         }
     },
     id: j++
 }

Let me know what are your thoughts and if I expect to see this feature available :)

jadell commented 10 years ago

The batch API is blocking as well, with the added downside that since you are sending all of the 1000s of batched requests at once, your PHP script may timeout or run out of memory receiving the results. With transactional Cypher, you can send multiple Cypher queries in multiple requests to the server, and all the requests will still run in the same transaction. This means you can fine-tune the groups of queries and responses for optimal performance. It is the best solution for sending many Cypher queries that all need to succeed or fail together. The Batch API should only be used if you need to send many REST operations that need to be run transactionally.

ArkeologeN commented 10 years ago

I may slightly or totally agree with you. As looking the performance I got over the cypher transactions, I think neo4j batch means is actual batch. It shard the cypher queries to new threads to run them in parallel? If is it so?

Your comments are logical. Can you please provide me a link where I can see that neo4j basically queue the request that are even placed for batch?

Cheers, :)

jadell commented 10 years ago

I wasn't able to find documentation that clearly states it is so. I found lots of responses from the Neo4j team that indicates it is the case. I posted a direct question on SO, so hopefully there will be a clear and direct answer.

http://stackoverflow.com/questions/23678848/are-batched-requests-sent-over-neo4j-rest-api-executed-in-parallel

jadell commented 10 years ago

The SO question has been answered by one of the Neo4j core team. Requests inside a batch are executed sequentially. I recommend using Cypher transactions if you have many Cypher queries/statements to make that all must succeed or fail as a group.

ArkeologeN commented 10 years ago

Alright, Sound's good! thanks @jadell