amark / gun

An open source cybersecurity protocol for syncing decentralized graph data.
https://gun.eco/docs
Other
18.05k stars 1.16k forks source link

How do you implement graph queries like openCypher or Gremlin on GunJS? #895

Open bionicles opened 4 years ago

bionicles commented 4 years ago

to model connected data as a medical research startup founder, I want to use a graph query language like openCypher or Gremlin; most specifically for pattern matching and graph-aided search algorithms (ex, indexing algolia results by pagerank or betweenness centrality)

The problem is there are no great options for such a thing for startups with limited budget. all the graph databases today, regardless of language, require the user to manually manage the capacity levels, which is a waste of money, crashes during peak demand, and pollutes the environment. Neo4j is licensed under GPLv3 which means if I put "docker run neo4j" in my CI/CD pipeline then I need to relicense my MIT project as GPLv3 and my contributors couldn't take the project and make proprietary corporate software (which I am OK with and don't want to limit) ... tinkerpop is neat but also super entrenched in the JVM which i'd rather avoid, and RedisGraph has openCypher and a fancy GraphBLAS linear algebra query runtime but their license says you cannot use their database in a "database product" ... unclear how that is even possible. AgensGraph runs on Postgres but only has a Java client. AnzoGraph claims to have a free commercial license but the legalese is super complicated and confusing and says you can't really use it, not sure why. Neptune advertises as "fully managed" then makes you manage your capacity with the AWS console. The only actual option for a serverless graph is Microsoft Azure CosmosDB Autopilot, which I guess I could try, but I'm concerned about getting sucked into yet another cloud provider.

Basically, graph databases today are built by people who are scared of competitors more than they want to help customers, so they put these complicated and limiting licenses which require a law degree to properly understand. openCypher and Gremlin are both licensed under Apache 2.0 but openCypher would require a complicated query runtime to translate cypher into datastore operations.

Gremlin is quite similar to GunJS and allows users to chain methods, so it's probably possible to add a "match" function to gun, which would find certain vertex-edge-vertex-edge patterns instead of just getting a specific key (we don't always know the key!), Is it possible to iterate over the nodes and filter paths? Another thing which would be handy would be some full text search with inverted indexes, and variable length traversals. Basically, I wish GunJS had some higher-level query options.

How hard do you think it would be to add some Gremlin-style pattern matching / search / traversal features to Gun?

Then there would be a serverless graph query language (note: graphql is not a graph query language IMO ... fake news marketing hype)

Dletta commented 4 years ago

Hey @bionicles,

If you write an abstraction on top of gun, you can indeed implement a pattern matching algorithm. https://github.com/dletta/visualgraph/ contains a js file abstraction.js. It creates an abstraction layer where you can post nodes and edges like in a labelled directed graph and allows a search that finds a combination of node-egde-node, based on some label. (Line 88 and onward)

The only requisite to use it is that the graph underneath is labeled (aka nodes are created using the abstraction)

kristianmandrup commented 2 years ago

I will start creating a cypher query engine for GUN