search

gchq / gaffer-experimental

Apache License 2.0
6 stars 7 forks source link

Implement TraversalStrategies in the TinkerPop implementation #80

Open p013570 opened 7 years ago

p013570 commented 7 years ago

The performance of our basic tinkerpop implementation is very poor. We need to make use TraversalStrategies to push down filtering to Gaffer/Accumulo.

Notes from @okram:

I would recommend binding it via the Java API. That is, implement the following interfaces in TinkerPop:

Graph: https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Graph.java
Vertex: https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Vertex.java
VertexProperty: https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/VertexProperty.java
Edge: https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Edge.java
Property: https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/Property.java

That is about 90% of the effort required. After that, to make things solid:

Connect it to the StructureStandardSuite and ProcessStandardSuite. You can see how this is done via TinkerGraph: https://github.com/apache/tinkerpop/blob/master/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/TinkerGraphStructureStandardTest.java and https://github.com/apache/tinkerpop/blob/master/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/process/TinkerGraphProcessStandardTest.java

Add a TraversalStrategy to use index lookups. I assume you have fast lookups for stuff like "get me all vertices with name 'marko'." If so, then you will need something like TinkerGraphStepStrategy (https://github.com/apache/tinkerpop/blob/master/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/process/traversal/strategy/optimization/TinkerGraphStepStrategy.java) where your GafferGraphStep implementation will use the HasContainers accordingly to do index lookups -- e.g. g.V().has('name','marko').

That is really all there is to it. I suspect a solid 1 week effort and you will then be able to use Gremlin to query your graph.

http://tinkerpop.apache.org/gremlin.html

Hope that helps.
n3101 commented 3 years ago

Tinkerpop was transferred from gaffer tools to experimental in 2016. This issue was left behind.