Unpickle serialized value from some previous process

[myio561]

Every example I have read starts with pickling some arbitrary object, then using the value returned by pickling method to unpickle that object. I don’t understand how to make my code on startup unpickle some serialized value created at some previous time where I don’t have a reference to the pickle object returned by the serialization method.

Can you point me in the right direction?

[phaller]

@myio561: For that you need to create a Pickle first, for example, a BinaryPickle based on a byte array or ByteBuffer, or a JSONPickle. Some relevant definitions:

// package scala.pickling.json
case class JSONPickle(value: String) extends Pickle

For binary:

// package scala.pickling.binary
object BinaryPickle {
  def apply(a: Array[Byte]): BinaryPickle = new BinaryPickleArray(a)
  def apply(a: BinaryInput): BinaryPickle = new BinaryInputPickle(a)
  def apply(a: InputStream): BinaryPickle = new BinaryInputPickle(new StreamInput(a))
  def apply(a: ByteBuffer): BinaryPickle = new BinaryInputPickle(new ByteBufferInput(a))
}

To unpickle, you can simply call p.unpickle[ExpectedType] where p is such a pickle.

[raam86]

You don't need a reference for the pickle object you just need a reference for the class of the object.

You can simply myObject.unpickle[MyObjectType]. Make sure you have pickling in scope

[myio561]

I must be missing a step somewhere. I tried:

import scala.beans.BeanProperty
case class FeatureUrl (
    @BeanProperty var group  : String,
    @BeanProperty var feature: String,
    @BeanProperty var url    : String
)

import scala.pickling.Defaults._
import scala.pickling.json._

val p = JSONPickle("[{\"group\":\"SAMPLE\",\"feature\":\"SPL-1\",\"url\":\"http://localhost:8080/SPL-1/sample\"}]")
var features = p.unpickle[Vector[FeatureUrl]]

When I run I get the error below on the line p.unpickle is called:

scala.MatchError: [{"group" : "SAMPLE", "feature" : "SPL-1", "url" : "http:\/\/localhost:8080\/SPL-1\/sample"}] (of class scala.util.parsing.json.JSONArray)
    at scala.pickling.json.JSONPickleReader$$anonfun$beginEntry$2.apply(JSONPickleFormat.scala:212)
    at scala.pickling.json.JSONPickleReader$$anonfun$beginEntry$2.apply(JSONPickleFormat.scala:203)
    at scala.pickling.PickleTools$class.withHints(Tools.scala:521)
    at scala.pickling.json.JSONPickleReader.withHints(JSONPickleFormat.scala:170)
    at scala.pickling.json.JSONPickleReader.beginEntry(JSONPickleFormat.scala:203)
    at scala.pickling.Unpickler$class.unpickleEntry(Pickler.scala:78)
    at scala.pickling.pickler.TravPickler$$anon$1.unpickleEntry(Iterable.scala:18)
    at scala.pickling.functions$.unpickle(functions.scala:11)
    at scala.pickling.UnpickleOps.unpickle(Ops.scala:23)
    at com.nextera.fim.servletMenu.Menu.<init>(Menu.scala:13)
    at com.nextera.fim.servletMenu.ServletMenu.menu$lzycompute(ServletMenu.scala:22)
    at com.nextera.fim.servletMenu.ServletMenu.menu(ServletMenu.scala:22)
    at com.nextera.fim.servletMenu.ServletMenu$$anonfun$2.apply(ServletMenu.scala:26)
    at com.nextera.fim.servletMenu.ServletMenu$$anonfun$2.apply(ServletMenu.scala:24)
    at org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$liftAction(ScalatraBase.scala:285)
    at org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:279)
    at org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:279)
    at org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:189)
    at com.nextera.fim.servletMenu.ServletMenu.withRouteMultiParams(ServletMenu.scala:15)
    at org.scalatra.ScalatraBase$class.invoke(ScalatraBase.scala:278)
    at com.nextera.fim.servletMenu.ServletMenu.org$scalatra$json$JsonSupport$$super$invoke(ServletMenu.scala:15)
    at org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:88)
    at org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:82)
    at org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:189)
    at com.nextera.fim.servletMenu.ServletMenu.withRouteMultiParams(ServletMenu.scala:15)
    at org.scalatra.json.JsonSupport$class.invoke(JsonSupport.scala:82)
    at com.nextera.fim.servletMenu.ServletMenu.invoke(ServletMenu.scala:15)
    at org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$8.apply(ScalatraBase.scala:253)
    at org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$8.apply(ScalatraBase.scala:251)
    at scala.Option.flatMap(Option.scala:171)
    at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:251)
    at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:250)
    at scala.collection.immutable.Stream.flatMap(Stream.scala:489)
    at org.scalatra.ScalatraBase$class.runRoutes(ScalatraBase.scala:250)
    at com.nextera.fim.servletMenu.ServletMenu.runRoutes(ServletMenu.scala:15)
    at org.scalatra.ScalatraBase$class.runActions$1(ScalatraBase.scala:175)
    at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply$mcV$sp(ScalatraBase.scala:187)
    at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:187)
    at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:187)
    at org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$cradleHalt(ScalatraBase.scala:205)
    at org.scalatra.ScalatraBase$class.executeRoutes(ScalatraBase.scala:187)
    at com.nextera.fim.servletMenu.ServletMenu.executeRoutes(ServletMenu.scala:15)
    at org.scalatra.ScalatraBase$$anonfun$handle$1.apply$mcV$sp(ScalatraBase.scala:126)
    at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:126)
    at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:126)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at org.scalatra.DynamicScope$class.withResponse(DynamicScope.scala:78)
    at com.nextera.fim.servletMenu.ServletMenu.withResponse(ServletMenu.scala:15)
    at org.scalatra.DynamicScope$$anonfun$withRequestResponse$1.apply(DynamicScope.scala:58)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at org.scalatra.DynamicScope$class.withRequest(DynamicScope.scala:69)
    at com.nextera.fim.servletMenu.ServletMenu.withRequest(ServletMenu.scala:15)
    at org.scalatra.DynamicScope$class.withRequestResponse(DynamicScope.scala:57)
    at com.nextera.fim.servletMenu.ServletMenu.withRequestResponse(ServletMenu.scala:15)
    at org.scalatra.ScalatraBase$class.handle(ScalatraBase.scala:125)
    at com.nextera.fim.servletMenu.ServletMenu.org$scalatra$servlet$ServletBase$$super$handle(ServletMenu.scala:15)
    at org.scalatra.servlet.ServletBase$class.handle(ServletBase.scala:53)
    at com.nextera.fim.servletMenu.ServletMenu.handle(ServletMenu.scala:15)
    at org.scalatra.ScalatraServlet$class.service(ScalatraServlet.scala:60)
    at com.nextera.fim.servletMenu.ServletMenu.service(ServletMenu.scala:15)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:568)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:462)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
    at java.lang.Thread.run(Thread.java:745)

When I debug, I found that inside JSONPickleFormat.beginEntry the fields member is NULL this is why the match fails.

[myio561]

I noticed the debug output from Pickle embeds type hints. I added those to my initialization and that fixed the exception in the previous post.

val p = JSONPickle("""{
                     |  "$type": "scala.Array[com.nextera.fim.servletMenu.FeatureUrl]",
                     |  "elems": [
                     |    {
                     |    "$type": "com.nextera.fim.servletMenu.FeatureUrl",
                     |    "group": "WPM",
                     |    "feature": "WPM-2348",
                     |    "url": "http:\/\/localhost:8100\/WPM-2348\/wpmClient"
                     |  }
                     |  ]
                     |}""".stripMargin)

However, my pickles are not persisted between sessions. Do I need to do something more like specify a path where pickles should be stored? When I test by initializing/creating the JSONPickle object above:

var features = p.unpickle[Array[FeatureUrl]]

features contains only the sample JSONPickle object.

[jsuereth]

@myio561 Yes, pickles are just in-memory datastructures (by default). You'll need to persist these somewhere if you want to persist them.