search

debug-ito / greskell

Haskell binding for Gremlin graph query language
https://hackage.haskell.org/package/greskell
27 stars 4 forks source link

How to specify its properties before adding a vertex? #5

Closed JeffreyBenjaminBrown closed 5 years ago

JeffreyBenjaminBrown commented 5 years ago

I figured out how to add a vertex:

runSideEffect :: GTraversal SideEffect () AVertex
             -> IO (Either SomeException [AVertex])
runSideEffect script = try go where
  go :: IO [AVertex]
  go =
    bracket (connect "localhost" 8182) close $ \client -> do
      result_handle <- submit client script Nothing
      fmap toList $ slurpResults result_handle

addPerson :: Int -> GTraversal SideEffect () AVertex
addPerson n = source "g" &
              sAddV "person"
              -- &. gProperty "suchness" (valueInt n)

Notice that the last line is commented out. If I run that, I get something like this:

Right [AVertex {avId = GValue {unGValue = GraphSON {gsonType = Just "g:Int64", gsonValue = GNumber 3.0}}, avLabel = "person", avProperties = PropertyMapList (PropertyMapGeneric (fromList []))}]

I hoped the commented-out line would add an Integer-valued property called "suchness" to the vertex. But if I uncomment that line, I get the same result -- the property map remains empty.

JeffreyBenjaminBrown commented 5 years ago

Further unsuccessful variations on that theme:

-- | I want this to add a person with "suchness" = n.
-- It adds a person, but with no properties.
addPerson :: Int -> GTraversal SideEffect () AVertex
addPerson n = source "g" &
              sAddV "person"
              &. x (gProperty "suchness" $ valueInt n)
  -- neither of these definitions of `x` does what I want
  where x = id
        -- x = gSideEffect

-- | I want this to set property "suchness" = 33
-- for every vertex in the graph. It changes nothing.
suchness33ForEverybody :: GTraversal SideEffect () AVertex
suchness33ForEverybody =
  ( liftWalk $ source "g" & sV []
    :: GTraversal SideEffect () AVertex )
  &. ( gProperty "suchness" 33
       :: Walk SideEffect AVertex AVertex )
debug-ito commented 5 years ago

Is this really an issue of greskell? Are you really sure that the Gremlin statement g.addV("person").property("suchness",3) should return vertex objects after the side-effect (i.e. adding the property) takes place?

JeffreyBenjaminBrown commented 5 years ago

Are you really sure that the Gremlin statement g.addV("person").property("suchness",3) should return vertex objects after the side-effect (i.e. adding the property) takes place?

I am not at all sure what it should return, no. Let's disregard what it should return, then, and use instead the following test:

findPeople :: GTraversal Transform () AVertex
findPeople =
  (source "g" & sV [] :: GTraversal Transform () AVertex)
  &. gHasLabel "person"

(The definitions of runSideEffect, addPerson and findPeople can all be found here.)

Starting from an empty graph, findPeople returns an empty list:

> runSideEffect $ liftWalk findPeople
Right []
>

Next, let's run addPerson, followed by findPeople:

> runSideEffect (addPerson 11) >> runSideEffect (liftWalk findPeople)
Right [AVertex {avId = GValue {unGValue = GraphSON {gsonType = Just "g:Int64",
gsonValue = GNumber 0.0}}, avLabel = "person", avProperties = PropertyMapList (PropertyMapGeneric (fromList []))}]
>

findPeople now returns a non-empty list, with a "person" in it -- great! But the "person" has no associated properties. And running suchness33ForEverybody has no discernible effect:

> runSideEffect suchness33ForEverybody >> runSideEffect (liftWalk findPeople)
Right [AVertex {avId = GValue {unGValue = GraphSON {gsonType = Just "g:Int64", gsonValue = GNumber 0.0}}, avLabel = "person", avProperties = PropertyMapList (PropertyMapGeneric (fromList []))}]
>
debug-ito commented 5 years ago 
> runSideEffect (addPerson 11) >> runSideEffect (liftWalk findPeople)

Hmm, the result of this statement actually looks weird.

Could you try the test WITHOUT greskell? For example, use gremlin.sh and run the equivalent commands in Gremlin. In addition, what data is returned from the Gremlin Server? Could you dump the data stream on WebSockets using, e.g., wireshark?

JeffreyBenjaminBrown commented 5 years ago

You bet! Once I get home from work. In the meantime, though, how would you yourself write a Greskell traversal that was intended to add a person with a property, or to adjust a property of some people?

debug-ito commented 5 years ago

I would write just the same way as your example.

I actually tried that (for example https://github.com/debug-ito/net-spider/blob/master/net-spider/src/NetSpider/Spider/Internal/Graph.hs#L83) and confirmed it worked. I usually use JanusGraph, though.

JeffreyBenjaminBrown commented 5 years ago

I haven't figured out how to use Wireshark yet. But I made some progress -- I found the property in GHCI, but still not the value.

Here's what I get from the Gremlin Console:

jeff@jeff-Inspiron-5567:~/installs/gremlin-console-3.4.2$ ./bin/gremlin.sh
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/home/jeff/installs/gremlin-console-3.4.2/lib/groovy-2.5.6-indy.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph
gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Configured localhost/127.0.0.1:8182
gremlin> :> g.V().hasLabel("person")
gremlin> :> g.addV("person").property("suchness",11)
==>v[0]
gremlin> :> g.V().hasLabel("person")
==>v[0]
gremlin>

And here's what's Gremlin Server has printed by the end of all that:

bash-4.4# /docker-entrypoint.sh conf/gremlin-server-jbb.yaml
[INFO] GremlinServer - 3.4.2
         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----

[INFO] GremlinServer - Configuring Gremlin Server from conf/gremlin-server-jbb.yaml
[INFO] MetricManager - Configured Metrics ConsoleReporter configured with report interval=180000ms
[INFO] MetricManager - Configured Metrics CsvReporter configured with report interval=180000ms to fileName=/tmp/gremlin-server-metrics.csv
[INFO] MetricManager - Configured Metrics JmxReporter configured with domain= and agentId=
[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics
[INFO] DefaultGraphManager - Graph [graph] was successfully configured via [conf/neo4j-jbb.properties].
[INFO] ServerGremlinExecutor - Initialized Gremlin thread pool.  Threads in pool named with pattern gremlin-*
[INFO] ServerGremlinExecutor - Initialized GremlinExecutor and preparing GremlinScriptEngines instances.
[INFO] ServerGremlinExecutor - Initialized gremlin-groovy GremlinScriptEngine and registered metrics
[INFO] ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[neo4jgraph[community single [/mnt/neo4j-graph]], standard]
[INFO] OpLoader - Adding the standard OpProcessor.
[INFO] OpLoader - Adding the session OpProcessor.
[INFO] OpLoader - Adding the traversal OpProcessor.
[INFO] TraversalOpProcessor - Initialized cache for TraversalOpProcessor with size 1000 and expiration time of 600000 ms
[INFO] GremlinServer - Executing start up LifeCycleHook
[INFO] Logger$info - Executed once at startup of Gremlin Server.
[INFO] GremlinServer - idleConnectionTimeout was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled
[INFO] GremlinServer - keepAliveInterval was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v3.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v3.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v3.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0
[INFO] AbstractChannelizer - Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0
[INFO] AbstractChannelizer - Configured application/vnd.graphbinary-v1.0 with org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1
[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 4 and boss thread pool of 1.
[INFO] GremlinServer$1 - Channel started at port 8182.

I wrote the following walk:

findPeopleProperties =
  ( source "g" & sV' [] )
  &. ( gHasLabel "person"
       >>> gProperties [] )

When I run it, I see suchness!

> runSideEffect $ liftWalk findPeopleProperties
Right [AVertexProperty {avpId = GValue {unGValue = GraphSON {gsonType = Just "g:Int64", gsonValue = GNumber -2.042816215e9}}, avpLabel = "suchness", avpValue = (), avpProperties = PropertyMapSingle (PropertyMapGeneric (fromList []))}]
>

So then I tried extracting the value, but nothing I do works. I tried all 4 variations on this:

findSuchnessValues =
  ( source "g" & sV' [] )
  &. ( gHasLabel "person"
       >>> gProperties []  -- maybe omit this line
       >>> gValues [] ) -- maybe use the empty list instead of ["suchness"]

If I don't include the gProperties line, I get Right [()]. If I include the gProperties line I get a Left like this:

> runSideEffect $ liftWalk findSuchnessValues
Left (ResponseError (ResponseMessage {requestId = fd53bd92-0320-4bc1-9dd7-ce6e5cc02008, status = ResponseStatus
{code = ScriptEvaluationError, message = "Properties on a vertex property is not supported", attributes = fromLi
st [("exceptions",Array [String "java.lang.UnsupportedOperationException"]),("stackTrace",String "java.lang.Unsu
pportedOperationException: Properties on a vertex property is not supported\n\tat org.apache.tinkerpop.gremlin.s
tructure.VertexProperty$Exceptions.metaPropertiesNotSupported(VertexProperty.java:107)\n\tat org.apache.tinkerpo
p.gremlin.neo4j.structure.trait.NoMultiNoMetaNeo4jTrait.getProperties(NoMultiNoMetaNeo4jTrait.java:146)\n\tat or
g.apache.tinkerpop.gremlin.neo4j.structure.Neo4jVertexProperty.properties(Neo4jVertexProperty.java:97)\n\tat org
.apache.tinkerpop.gremlin.structure.Element.values(Element.java:107)\n\tat org.apache.tinkerpop.gremlin.process.
traversal.step.map.PropertiesStep.flatMap(PropertiesStep.java:63)\n\tat org.apache.tinkerpop.gremlin.process.tra
versal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49)\n\tat org.apache.tinkerpop.gremlin.process.tra
versal.step.util.AbstractStep.hasNext(AbstractStep.java:143)\n\tat org.apache.tinkerpop.gremlin.process.traversa
l.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)\n\tat org.apache.tinkerpop.gremlin.proce
ss.traversal.step.map.MapStep.processNextStart(MapStep.java:36)\n\tat org.apache.tinkerpop.gremlin.process.trave
rsal.step.util.AbstractStep.hasNext(AbstractStep.java:143)\n\tat org.apache.tinkerpop.gremlin.process.traversal.
util.DefaultTraversal.hasNext(DefaultTraversal.java:196)\n\tat org.apache.tinkerpop.gremlin.server.op.AbstractOp
Processor.handleIterator(AbstractOpProcessor.java:96)\n\tat org.apache.tinkerpop.gremlin.server.op.AbstractEvalO
pProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:275)\n\tat org.apache.tinkerpop.gremlin.groovy.e
ngine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:277)\n\tat java.util.concurrent.FutureTask.run(FutureTa
sk.java:266)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concu
rrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec
utor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java
.lang.Thread.run(Thread.java:748)\n")]}, result = ResponseResult {resultData = GValue {unGValue = GraphSON {gson
Type = Nothing, gsonValue = GNull}}, meta = fromList []}}))
debug-ito commented 5 years ago

g.V().hasLabel("person").properties().values() returns values of meta-properties of vertex properties, which is NOT values of vertex properties. The .properties step should not be included.

As for the last error message, it looks like your Gremlin Server doesn't support meta-properties on vertex properties.

Give up greskell for a while. It complicates debugging. Keep testing your Gremlin Server with gremlin.sh and dump the WebSocket communicaiton.

JeffreyBenjaminBrown commented 5 years ago

Captured packets + detailed readme available here. Thanks, Toshio.

debug-ito commented 5 years ago

Thanks.

Sorry, I forgot to tell you, but gremlin.sh seems to use gryo format by default. You have to configure it to use json format somehow.

debug-ito commented 5 years ago

To do that, edit serializer field of remote.yaml like

serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 }

I'm not sure if that is the right serializer. See https://tinkerpop.apache.org/javadocs/current/full/org/apache/tinkerpop/gremlin/driver/ser/package-summary.html for other serializers.

JeffreyBenjaminBrown commented 5 years ago

Updated packet capture and README

Thanks, Toshio!

debug-ito commented 5 years ago

Thanks.

So, the dump reveals that the server does not send back the property keys and values in the first place (see packet 41). Usually a Vertex JSON object should have an attribute called properties, which keeps a JSON object of all properties of the Vertex. However, the Vertex objects in your dump don't have properties attribute.

I guess this is a problem of graph implementation (Neo4JGraph?) Try other graph implementations to see any differences, or search bug trackers of the graph implementation for any clue on this problem.

debug-ito commented 5 years ago

Closed because this is not an issue of greskell. I opened #6 as the issue of greskell regarding this problem.

@JeffreyBenjaminBrown if you make any progress on this problem, feel free to comment on this issue. Maybe I can help you, and it can help to figure out how we should approach to #6.