search

JanusGraph / janusgraph

JanusGraph: an open-source, distributed graph database
https://janusgraph.org
Other
5.27k stars 1.16k forks source link

Kryo registrator options for Spark #1194

Open pluradj opened 6 years ago

pluradj commented 6 years ago

This came up during a documentation review:

https://github.com/JanusGraph/janusgraph/blob/0.2/janusgraph-hadoop-parent/janusgraph-hadoop-core/src/test/resources/cassandra-read.properties#L30 Should we use:

spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator

Or:

spark.kryo.registrator=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator

JanusGraphKryoRegistrator only registers a serializer for Geoshape, while the TinkerPop GryoRegistrator handles many different classes. JanusGraphKryoRegistrator does not extend GryoRegistrator.

We need some clarity on how to configure the Kryo registrator to cover all required TinkerPop and JanusGraph classes. Also will need updates on the documentation and the properties files in the distribution.

zpi1223 commented 6 years ago

@pluradj I see，tks; I want to see the janusgraph-users for google is very inconvenient, Is it possible to open a channel on jira?

pluradj commented 6 years ago

@AaronCH5 I will bring up your concerns on the developer list. The JanusGraph project doesn't use Jira.