search

thinkaurelius / titan

Distributed Graph Database
http://titandb.io
Apache License 2.0
5.24k stars 1.01k forks source link

titan09 gremlin-server serializing issue on session OpProcessor #1094

Closed dmill-bz closed 9 years ago

dmill-bz commented 9 years ago

Hey,

I'm running ./bin/titan.sh -v start built from titan09. (no change in configuration or anything)

When using the session OpProcessor in combination with the GraphSONMessageSerializerV1d0 I get an error when i run g.addV('name', 'john') (see trace at the end of the post)

This works fine using the console (I'm guessing GryoMessageSerializer ?) This also works fine in vanilla gremlin-server M9-rc3.

It looks like titan is using AbstractGraphSONMessageSerializerV1d0 to try and serialize the message.

Also, even though I stated the OpProcessor as session I'm not getting the usual : 

[INFO] Session - New session established for 43aec405-1919-4e66-a87c-3768efd08562

Should I make another issue for this? Or is it normal?

40137 [gremlin-server-worker-1] INFO  org.apache.tinkerpop.gremlin.server.op.OpLoader  - Adding the standard OpProcessor.
40139 [gremlin-server-worker-1] INFO  org.apache.tinkerpop.gremlin.server.op.OpLoader  - Adding the control OpProcessor.
40141 [gremlin-server-worker-1] INFO  org.apache.tinkerpop.gremlin.server.op.OpLoader  - Adding the session OpProcessor.
41126 [gremlin-server-worker-1] WARN  org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0  - Response [ResponseMessage{requestId=b0d04318-3b6f-4084-9d43-52f0f13c43fe, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[4248]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
41131 [gremlin-server-worker-1] WARN  org.apache.tinkerpop.gremlin.server.handler.WsGremlinResponseEncoder  - The result [ResponseResult{data=[v[4248]], meta={}}] in the request b0d04318-3b6f-4084-9d43-52f0f13c43fe could not be serialized and returned.
org.apache.tinkerpop.gremlin.driver.ser.SerializationException: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.IllegalStateException) (through reference chain: java.util.ArrayList[0]->com.thinkaurelius.titan.graphdb.relations.RelationIdentifier["inVertexId"])
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.serializeResponseAsBinary(AbstractGraphSONMessageSerializerV1d0.java:116)
    at org.apache.tinkerpop.gremlin.server.handler.WsGremlinResponseEncoder.encode(WsGremlinResponseEncoder.java:65)
    at org.apache.tinkerpop.gremlin.server.handler.WsGremlinResponseEncoder.encode(WsGremlinResponseEncoder.java:45)
    at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:89)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:691)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:626)
    at org.apache.tinkerpop.gremlin.server.handler.IteratorHandler.write(IteratorHandler.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633)
    at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32)
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:908)
    at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:960)
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:893)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:380)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.IllegalStateException) (through reference chain: java.util.ArrayList[0]->com.thinkaurelius.titan.graphdb.relations.RelationIdentifier["inVertexId"])
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:232)
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:197)
    at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:186)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:640)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:152)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:46)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers.serializerVertexProperty(GraphSONSerializers.java:333)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers.access$000(GraphSONSerializers.java:55)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers$VertexJacksonSerializer.writeProperties(GraphSONSerializers.java:213)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers$VertexJacksonSerializer.ser(GraphSONSerializers.java:190)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers$VertexJacksonSerializer.serialize(GraphSONSerializers.java:172)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializers$VertexJacksonSerializer.serialize(GraphSONSerializers.java:160)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:100)
    at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:21)
    at com.fasterxml.jackson.databind.ser.std.AsArraySerializerBase.serialize(AsArraySerializerBase.java:183)
    at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:46)
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:235)
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:205)
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:197)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:114)
    at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:2811)
    at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:2292)
    at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.serializeResponseAsBinary(AbstractGraphSONMessageSerializerV1d0.java:107)
    ... 16 more
Caused by: java.lang.IllegalStateException
    at com.google.common.base.Preconditions.checkState(Preconditions.java:134)
    at com.thinkaurelius.titan.graphdb.relations.RelationIdentifier.getInVertexId(RelationIdentifier.java:69)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.get(BeanPropertyWriter.java:726)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:506)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:632)
    ... 35 more
spmallette commented 9 years ago

I'm not expecting to see this error on the titan09 branch this point - I assume you built this from there?

dmill-bz commented 9 years ago

Yes I built again after your changes on the configuration files (skipped the tests though). I did wonder at one point if it was using the correct TP3 jars as I never had to manually configure anything. I just built TP3-snapshot and then titan09 and it simply worked (I assumed maven kept some trace of the TP3 files in cache or something)

On the off chance that I've been screwing up the build can you run me through the process quickly?

spmallette commented 9 years ago

are you building the distribution zip file in titan or taking some other approach (e.g. building both and then doing a manual install of titan into gremlin server)?

dmill-bz commented 9 years ago

Here's what I've been doing (hopefully correct):

mvn -v

Apache Maven 3.0.5
Maven home: /usr/share/maven
Java version: 1.8.0_45, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-51-generic", arch: "amd64", family: "unix"
git clone --branch titan09 --depth=1 https://github.com/thinkaurelius/titan.git
cd titan
mvn clean install -DskipTests=true
bin/titan.sh -v start
spmallette commented 9 years ago

hmmm - i'm not sure if your build of titan is good. i don't run gremlin server from there like that - i run from the distribution. to generate the distribution do:

mvn clean install -DskipTests=true -Paurelius-release -Dgpg.skip=true

which can be run from the titan-dist/titan-dist-hadoop-1 directory. then look in the target directory there and unzip the package to your system and run from there. obviously you should make sure that you have TP3 built locally prior to that.

please give that a try and let's see if that fixes anything.

dmill-bz commented 9 years ago

Ok, I just did and it still fails with the same error.

spmallette commented 9 years ago

:rage4:

dmill-bz commented 9 years ago

:sob:

spmallette commented 9 years ago

i think i know what's wrong - bad configuration for serializers - update as follows:

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { useMapperFromGraph: graph }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { useMapperFromGraph: graph }}

please see: http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_serialization

of course those settings should be there by default in the packaged yaml - updating that now.

spmallette commented 9 years ago

Note that i just updated those configs by adding commas between the classname entry and the config entry.

dmill-bz commented 9 years ago

Awesome!! That works and all the transaction tests pass for the driver. Quick question: we can only reference one graph in the conf?

spmallette commented 9 years ago

good - glad that's working.

at the moment, you can only reference one graph. we need to implement some kind of chaining mechanism in IoRegistry to allow multiple to be assigned:

https://issues.apache.org/jira/browse/TINKERPOP3-656

There is a workaround - but it would involve a custom message serializer.