How to serialize a ptmodel

[napdada]

hi！I want to use the pytorch model in a distributed environment, but many classes in djl do not implement the Serializable interface.

Inheriting classes and then implementing Serializable interface will involve many other classes that still cannot be serialized, and the whole process is very troublesome, is there an easy way to do this?

error example： `Caused by: java.io.NotSerializableException: ai.djl.pytorch.engine.PtModel Serialization stack:

• object not serializable (class: ai.djl.pytorch.engine.PtModel, value: ai.djl.pytorch.engine.PtModel@411a5965)
• field (class: model.Encoder, name: model, type: interface ai.djl.Model)
• object (class model.Encoder, model.Encoder@53ed80d3)
• field (class: absfunc.vertex.UpdateFeat, name: encoder, type: class model.Encoder)
• object (class absfunc.vertex.UpdateFeat, )
• element of array (index: 1)
• array (class [Ljava.lang.Object;, size 3)
• field (class: java.lang.invoke.SerializedLambda, name: capturedArgs, type: class [Ljava.lang.Object;)
• object (class java.lang.invoke.SerializedLambda, SerializedLambda[capturingClass=class org.apache.spark.graphx.impl.GraphImpl, functionalInterfaceMethod=scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;, implementation=invokeStatic org/apache/spark/graphx/impl/GraphImpl.$anonfun$mapVertices$1:(Lorg/apache/spark/graphx/impl/GraphImpl;Lscala/Function2;Lscala/reflect/ClassTag;Lorg/apache/spark/graphx/impl/ShippableVertexPartition;)Lorg/apache/spark/graphx/impl/ShippableVertexPartition;, instantiatedMethodType=(Lorg/apache/spark/graphx/impl/ShippableVertexPartition;)Lorg/apache/spark/graphx/impl/ShippableVertexPartition;, numCaptured=3])
• writeReplace data (class: java.lang.invoke.SerializedLambda)
• object (class org.apache.spark.graphx.impl.GraphImpl$$Lambda$1500/1168942230, org.apache.spark.graphx.impl.GraphImpl$$Lambda$1500/1168942230@7978e022)
• element of array (index: 0)
• array (class [Ljava.lang.Object;, size 1)
• field (class: java.lang.invoke.SerializedLambda, name: capturedArgs, type: class [Ljava.lang.Object;)
• object (class java.lang.invoke.SerializedLambda, SerializedLambda[capturingClass=class org.apache.spark.graphx.impl.VertexRDDImpl, functionalInterfaceMethod=scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;, implementation=invokeStatic org/apache/spark/graphx/impl/VertexRDDImpl.$anonfun$mapVertexPartitions$1:(Lscala/Function1;Lscala/collection/Iterator;)Lscala/collection/Iterator;, instantiatedMethodType=(Lscala/collection/Iterator;)Lscala/collection/Iterator;, numCaptured=1])
• writeReplace data (class: java.lang.invoke.SerializedLambda)
• object (class org.apache.spark.graphx.impl.VertexRDDImpl$$Lambda$1501/1769042905, org.apache.spark.graphx.impl.VertexRDDImpl$$Lambda$1501/1769042905@7fb53256) at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41) at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47) at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:101) at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:413) ... 13 more`

Thanks！

[frankfliu]

@napdada Please take a look our spark examples about how DJL works in Spark: https://github.com/deepjavalibrary/djl-demo/tree/master/apache-spark