search

spring-projects / spring-data-neo4j

Provide support to increase developer productivity in Java when using Neo4j. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.
http://spring.io/projects/spring-data-neo4j
Apache License 2.0
822 stars 619 forks source link

Poor Save Performance Compared to Neo4j OGM #2636

Open jrsperry opened 1 year ago

jrsperry commented 1 year ago

When using non-simplistic graphs (graphs with multiple relationships between nodes or more connected graphs), the Neo4j OGM vastly outperforms SDN in saving data to a graph (tested using sdn 6.3.3 and sdn 7.0.0 and comparing against Neo4j OGM 3.2.38, also tested both Neo4j DB 4.4 and 5.2). In graphs consisting of only nodes, or nodes with 1 relationship the performance is comparable.

Here are links to the 2 example projects comparing the projects. On my laptop (M1 Pro CPU) the SDN is ~4 times slower than the OGM. In my work with a more complex schema it's ~10 times slower. The SDN example project has 2 branches, with master using SDN 6.3.3 and the other using SDN 7.0.0. In either project there's a test you can run and logging to show how long it takes to load an object after a warmup period.

SDN Example OGM Example

mrksph commented 1 year ago

Hi, thank you so much for raising this issue. I was expecting some difference in perfomance using SDN and OGM but I wouldn't say it is that big.

I will give it at try after holidays.

meistermeier commented 1 year ago

I have seen this and of course we want to improve your situation. Sorry for not interacting with the issue until now. We are currently working on a mechanism that fetches relationships from the database to diff before doing the superfluous remove/create statements. First numbers are promising (1/2 up to 1/4 of the current time it takes for 100 and 1000 relationships, improves with the amount of relationships). Will keep you updated when a snapshot will be available to get your feedback on this.

meistermeier commented 1 year ago

Update: I am still on this topic but have to rethink a few bits in there. There were some test failures and after correcting those the alternative version is more or less equal up to 10000 relationships 😞 Also, I cannot cover every type of relationship right now (dynamic relationships). On the positive side: It creates less subsequent queries but puts the load on the database. Sorry that I cannot report any win here right now, but wanted to keep you updated on this topic.

jrsperry commented 11 months ago

Any update?

meistermeier commented 9 months ago

Unfortunately as mentioned beginning of this year, the prefetching did not succeed. I am looking forward to try another approach in the next weeks (let's be honest, months) that might even eliminate the need to process all relationships. Cannot tell more because at the moment because I don't want to promise anything.