Open jexp opened 7 years ago
I'm not convinced that using memory to do this is the right thing. You are going to run into heap issues and threading challenges. I'm thinking of doing this by using a special node in the DB to cache node ids
Definitely not, caching nodes or id's is no problem. A node-proxy is just a thin wrapper around the long-id. so it's at most 16+8 bytes.
OK. We are going to fork the repo and work on this. Will send a pull request when initial work in place.
Also add a cache for certain node or relationship properties -> node/rel-id -> property value. Potentially as optimized column store where repeated values are only stored once or with fewer bits
As Cypher now uses caffeine, we should also use that, with TTL and dynamic cache rebuilds (if needed)
Can you expand on cyperh's use of caffeine? I must have missed that
Sent from my iPhone
On Jan 9, 2017, at 19:18, Michael Hunger notifications@github.com wrote:
Also add a cache for certain node or relationship properties -> node/rel-id -> property value. Potentially as optimized column store where repeated values are only stored once or with fewer bits
As Cypher now uses caffeine, we should also use that, with TTL and dynamic cache rebuilds (if needed)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
@crichey it uses it for its internal query -> query-plan cache. Just means we don't have to add an extra library dependency
@crichey any updates on your work, otherwise I'd love to look at this in the next weeks.
We are doing some internal testing today and will review tomorrow. I will provide update then.
From: Michael Hunger [mailto:notifications@github.com] Sent: Wednesday, January 18, 2017 11:44 AM To: neo4j-contrib/neo4j-apoc-procedures neo4j-apoc-procedures@noreply.github.com Cc: Clark D. Richey, Jr. clark@clarkrichey.com; Mention mention@noreply.github.com Subject: Re: [neo4j-contrib/neo4j-apoc-procedures] add a static cache for nodes based on properties or other criteria (#226)
@crichey https://github.com/crichey any updates on your work, otherwise I'd love to look at this in the next weeks.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/226#issuecomment-273529507 , or mute the thread https://github.com/notifications/unsubscribe-auth/AANEYHaYxdPxUpje2K0rjd7ulE7s3GBmks5rTkFTgaJpZM4LHN5_ . https://github.com/notifications/beacon/AANEYEOb849TapoCW0K5zo3uwd4T7rtAks5rTkFTgaJpZM4LHN5_.gif
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
Is this still in progress or not? Otherwise we can work on it
for frequenly used nodes and relationships, create an in-memory cache (index) that puts them into a (optionally sorted) map by property
a named cache lives in a static field
map property to node-id or node-proxy
need to be multi-thread-save
manage TTL/refresh (caffeine) ?
build cache from cypher statement
add to cache dynamically
make sure the cache can be configured to be sorted by value, so that it allows range-queries for the property and also returns the nodes / relationship in sort-order
apoc.cache.add(key, node | relationship) apoc.cache.add.query("match (n) return n.foo as value, n as node")
apoc.cache.nodes.for(key,value) -> stream of nodes apoc.cache.rels.for(key, value) -> stream of relationships apoc.cache.nodes.between(key, from, to) -> return stream values in sort order