Class is not registered: com.lambdazen.bitsy.store.VertexBean

[giovibal]

With TP 3.3.3 and private java client, we obtain this exception when try to load Vertices:

org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Class is not registered: com.lambdazen.bitsy.store.VertexBean
Note: To register this class use: kryo.register(com.lambdazen.bitsy.store.VertexBean.class);
        at org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV3d0.serializeResponseAsBinary(AbstractGryoMessageSerializerV3d0.java:193)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(AbstractOpProcessor.java:266)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:146)
        at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:252)
        at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:274)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Class is not registered: com.lambdazen.bitsy.store.VertexBean
Note: To register this class use: kryo.register(com.lambdazen.bitsy.store.VertexBean.class);
        at org.apache.tinkerpop.shaded.kryo.Kryo.getRegistration(Kryo.java:484)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.AbstractGryoClassResolver.writeClass(AbstractGryoClassResolver.java:110)
        at org.apache.tinkerpop.shaded.kryo.Kryo.writeClass(Kryo.java:514)
        at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:619)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedKryoAdapter.writeClassAndObject(ShadedKryoAdapter.java:49)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedKryoAdapter.writeClassAndObject(ShadedKryoAdapter.java:24)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoSerializersV3d0$VertexSerializer.write(GryoSerializersV3d0.java:133)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.GryoSerializersV3d0$VertexSerializer.write(GryoSerializersV3d0.java:130)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedSerializerAdapter.write(ShadedSerializerAdapter.java:44)
        at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:625)
        at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)
        at org.apache.tinkerpop.shaded.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
        at org.apache.tinkerpop.shaded.kryo.Kryo.writeClassAndObject(Kryo.java:625)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedKryoAdapter.writeClassAndObject(ShadedKryoAdapter.java:49)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedKryoAdapter.writeClassAndObject(ShadedKryoAdapter.java:24)
        at org.apache.tinkerpop.gremlin.driver.ser.ResponseMessageGryoSerializer.write(ResponseMessageGryoSerializer.java:45)
        at org.apache.tinkerpop.gremlin.driver.ser.ResponseMessageGryoSerializer.write(ResponseMessageGryoSerializer.java:34)
        at org.apache.tinkerpop.gremlin.structure.io.gryo.kryoshim.shaded.ShadedSerializerAdapter.write(ShadedSerializerAdapter.java:44)
        at org.apache.tinkerpop.shaded.kryo.Kryo.writeObject(Kryo.java:531)
        at org.apache.tinkerpop.gremlin.driver.ser.AbstractGryoMessageSerializerV3d0.serializeResponseAsBinary(AbstractGryoMessageSerializerV3d0.java:177)
        ... 10 more

this is my java client code, the exception is insiede "iterator().next()" and came from gremlin server:

public void read(ResultSet vertices) {
        // read all vertices
        JsonArray verticesJson = new JsonArray();
        if(vertices!=null) {
            vertices.iterator().forEachRemaining(vertexRS -> {
                Vertex vertex = vertexRS.getVertex();
                JsonObject vertexJson = new JsonGraphVertex().read(vertex).json();
                verticesJson.add(vertexJson);
            });
        }

I've tried some serialization config, but without success.

[lambdazen]

Hi Giovanni,

Are you running Bitsy in the same VM or in a Gremlin Server connected via WebSockets?

I have seen an error like this in issue #7, but it went away once we registered BitsyIoRegistryV3d0 in the server configuration.

See:

• https://github.com/lambdazen/bitsy/wiki#using-bitsy-in-gremlin-console-with-gremlin-server
• https://github.com/lambdazen/bitsy/blob/master/src/test/resources/gremlin-server/gremlin-server-bitsy.yaml
• https://github.com/lambdazen/bitsy/blob/c51e3a23c75c92ac63b689d77a5bfac0bb4c4f3e/src/main/java/com/lambdazen/bitsy/BitsyIoRegistryV3d0.java

If you have already registered past these steps, I wonder if the Gremlin console behaves differently from a plain Java interface.

Regards,

Sridhar.

[giovibal]

No, different vm, one for gremlin-server-3.3.3 + bitzy, another with my java client. I've solved my problem with a "customized" version of bitsy, i've added register calls for VertexBean and EdgeBean classes, reinstalled bitsy to gremlin-server, and it works.

This is the new constructor of "BitsyIoRegistryV3d0" class:

  private BitsyIoRegistryV3d0() {
    register(GryoIo.class, UUID.class, new UUIDGryoSerializer());
    register(GryoIo.class, com.lambdazen.bitsy.store.VertexBean.class, new UUIDGryoSerializer());
    register(GryoIo.class, com.lambdazen.bitsy.store.EdgeBean.class, new UUIDGryoSerializer());
  }

[giovibal]

I will add a pull request asap

[lambdazen]

Thanks! I am currently traveling and will be back mid next week.

I will release 3.0.4 with your patch by the end of next week, and close this issue.

Thanks again for your help in resolving this.

Regards,

Sridhar.

[cstamas]

Just out of curiosity, the change does not look right to me. You register UUIDGryoSerializer for both, EdgeBean and VertexBean. True, these two extend UUID, but they have much more properties added too. Did you verify, that all the data comes thru the pipe with this change? Isn't the set serializer (that handles MSB and LSB only) eating up the edge or vertex properties?

Simply put, assuming this was tested, the change does not look right to me, still.

[giovibal]

I agree with you, I've used UUIDGryoSerializer just because VertexBean and EdgeBean extends UUID and it solved my bug with minor changes to bitsy codebase. I think that VertexBean is serialized only because there is a public "getter" in BitsyVertex, and not for any other use. I'm not an expert about gremlin serialization and wire protocol, so you can consider this patch as a "workaround" to permit the deserialization of an entire vertex from gremlin server to client.

I tried also to refactor BitsyVertex to not contain "toBean" methods and to not depends on "VertexBean" anymore, in my opinion is more correct to decouple VertexBean from BitsyVertex because they are in different application layers.

[lambdazen]

OK. Makes sense. The stuff that gets serialized needs to be carefully designed to work with Kryo.

My guess is that the Gremlin Console has some code to copy over the data to other Vertex and Edge implementations, otherwise the tests wouldn't have worked.

Can you point me to an example with a Java client that talks to a Gremlin server? I will make sure all the unit test run with this configurarion before making the release.

Regards,

Sridhar.

[giovibal]

This is a "simplified" version of our client code who read all vertices of a graph:

ResultSet vertices = client.submit("g.V()", params);
JsonArray verticesJson = new JsonArray();
if(vertices!=null) {
    vertices.iterator().forEachRemaining(vertexRS -> {
        Vertex vertex = vertexRS.getVertex();
        JsonObject vertexJson = new JsonObject();
        Set<String> props = vertex.keys();
        for (String prop : props) {
            vertexJson.put(prop, vertex.property(prop).value().toString());
        }
        verticesJson.add(vertexJson);
    });
}

[lambdazen]

Thanks. I will look at this over the weekend.

[giovibal]

I've tried to detach VertexBean class from BitsyVertex class, but the serialization exception persists. Why VertexBean need to be serialized from gremlin-server when there is a query of type "g.V()" ?

[giovibal]

Any updates ?

[lambdazen]

Apologies for the delay... work has been a little hectic. I will get back to you this week.

[lambdazen]

Found the culprit... https://github.com/lambdazen/bitsy/blob/6cd4d1b23568ae03c11f8d90eaa54dfcdeb3b662/src/main/java/com/lambdazen/bitsy/store/VertexBean.java#L42

The Gremlin/Kryo serializers ended up with these VertexBeans and EdgeBeans because of an optimization whereby the beans are themselves used as IDs to reduce expensive Object.equals().

Only the IDs are serialized directly by Gremlin/Kryo directly. All other classes are converted to reference classes like these:

• org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertex
• org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge
• org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceVertexProperty
• and so on...

Hence your original fix pretty much work as is. Thanks!

The only thing is that you'll have to use register the IO registry in the server yaml as follows:

serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [com.lambdazen.bitsy.BitsyIoRegistryV3d0] }}            # application/vnd.gremlin-v3.0+gryo

and in the client (if using Java) as follows:

GryoMapper.Builder kryoBuilder = GryoMapper.build().addRegistry(BitsyIoRegistryV3d0.instance());
MessageSerializer serializer = new GryoMessageSerializerV3d0(kryoBuilder);
Cluster.Builder builder = Cluster.build();
builder.addContactPoint("localhost"); // or other domain/IP
builder.port(8182); // or other port
builder.serializer(serializer);
Cluster cluster = builder.create();
DriverRemoteConnection remote = DriverRemoteConnection.using(cluster);
GraphTraversalSource g = EmptyGraph.instance().traversal().withRemote(remote);

// Now use g for traversals
g.V().forEachRemaining(new Consumer<Vertex>() {
  public void accept(Vertex vertex) {
    System.out.println("Found " + vertex.id() + " with keys " + vertex.keys() + " of class " + vertex.getClass());
  }
});

I will release the next version tonight after some code cleanup.

[lambdazen]

I up'd the version 3.1.0 to account for changes to Gremlin, Jackson and NESS (thank you @cstamas )

https://oss.sonatype.org/content/groups/public/com/lambdazen/bitsy/bitsy/3.1.0/