Cannot deserialize `None` values in a tuple

[negokaz]

Deserializing a tuple that contains Some values succeeds. However, a MismatchedInputException will be thrown during deserialization If the tuple contains None values.

Version

• Scala 2.13.6
• jackson-databind 2.12.3
• jackson-module-scala 2.12.3

Actual behavior

The test case "deserialize OptionalTupleHolder that has 'None' values" fails.

package com.example

import com.fasterxml.jackson.databind.json.JsonMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import org.scalatest.funsuite.AnyFunSuite
import org.scalatest.matchers.should.Matchers

object JacksonScalaSpec {
  final case class OptionalTupleHolder(tuple: (Option[Int], Option[String]))
}

class JacksonScalaSpec extends AnyFunSuite with Matchers {
  import JacksonScalaSpec._

  private val mapper =
    JsonMapper
      .builder()
      .addModule(DefaultScalaModule)
      .build()

  println(mapper.version())

  test("deserialize OptionalTupleHolder that has 'Some' values") {
    val obj  = OptionalTupleHolder((Option(1), Option("2")))
    val json = mapper.writeValueAsString(obj)
    println(json) // {"tuple":[1,"2"]}
    val deserialized = mapper.reyadValue(json, classOf[OptionalTupleHolder])
    println(deserialized)

    obj should be(deserialized)
    // pass
  }

  test("deserialize OptionalTupleHolder that has 'None' values") {
    val obj  = OptionalTupleHolder((Option.empty, Option.empty))
    val json = mapper.writeValueAsString(obj)
    println(json) // {"tuple":[null,null]}
    val deserialized = mapper.readValue(json, classOf[OptionalTupleHolder])
    // throws com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.lang.String` from Null value (token `JsonToken.VALUE_NULL`)
    println(deserialized)

    obj should be(deserialized)
  }
}

Details of MismatchedInputException

Cannot deserialize value of type `java.lang.String` from Null value (token `JsonToken.VALUE_NULL`)
 at [Source: (String)"{"tuple":[null,null]}"; line: 1, column: 16] (through reference chain: com.example.JacksonScalaSpec$OptionalTupleHolder["tuple"])
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.lang.String` from Null value (token `JsonToken.VALUE_NULL`)
 at [Source: (String)"{"tuple":[null,null]}"; line: 1, column: 16] (through reference chain: com.example.JacksonScalaSpec$OptionalTupleHolder["tuple"])
    at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
    at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1601)
    at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1375)
    at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1280)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:73)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
    at com.fasterxml.jackson.module.scala.deser.OptionDeserializer.deserialize(OptionDeserializerModule.scala:61)
    at com.fasterxml.jackson.module.scala.deser.OptionDeserializer.deserialize(OptionDeserializerModule.scala:11)
    at com.fasterxml.jackson.module.scala.deser.TupleDeserializer.$anonfun$deserialize$1(TupleDeserializerModule.scala:48)
    at scala.collection.immutable.Vector1.map(Vector.scala:1886)
    at scala.collection.immutable.Vector1.map(Vector.scala:375)
    at com.fasterxml.jackson.module.scala.deser.TupleDeserializer.deserialize(TupleDeserializerModule.scala:45)
    at com.fasterxml.jackson.module.scala.deser.TupleDeserializer.deserialize(TupleDeserializerModule.scala:10)
    at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:565)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:449)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
    at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3548)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3516)
    at com.example.JacksonScalaSpec.$anonfun$new$2(JacksonScalaSpec.scala:38)

Expected behavior

The json {"tuple":[null,null]} is deserialized to OptionalTupleHolder((None, None)) without any Exceptions.

[pjfanning]

I'm currently travelling and when I get back, I have higher priority items to look at. Using tuples for data binding is supported but not encouraged. Could you use a case class? The JSON will look a little different but case classes are the Scala norm for data binding. You could also try testing with a newer version of this lib. There are also numerous alternative libs out there if this one doesn't suit your needs.

[cowtowncoder]

In particular, trying with 2.14.2 (or 2.15.0-rc2) would be easy step to see that problem still occurs. 2.12 is bit old version.

[negokaz]

@pjfanning Thank you for your reply. I understand your situation. We will try to use case class instead of tuples.

[pjfanning]

I haven't check the deserialization code yet but serialization seems to work (which is a start) - https://github.com/FasterXML/jackson-module-scala/commit/1741f14197ff3c59c14b2cfc8bc73286ba124636

[pjfanning]

@cowtowncoder I tried this deserialization case and the issue is happening in jackson-databind StringDeserializer.

This test case works with scala.Option (this scala class is very similar to java.lang.Optional. But with scala.Option, jackson-module-scala creates a jackson-databind StringDeserializer to handle the deserialization of the underlying value.

jackson-databind StringDeserializer blows up when it sees null. I'd happily accept back Java null in this case, because an Option can handle nulls. Is there any way to create a jackson-databind StringDeserializer that accepts nulls?

    at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1280)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:73)
    at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)

[pjfanning]

@cowtowncoder I added this check in the jackson-module-scala code (https://github.com/FasterXML/jackson-module-scala/commit/eb55d7a3e432390c57a4e785556953ea2e534b7a) so that I can catch the fact the JSON value is null before invoking the inner deserializer.

[cowtowncoder]

@pjfanning Contract with jackson-databind is that the "parent" deserializer is to handle null tokens: so, for example, StringDeserializer never sees null but only non-null tokens.

In case of POJO properties, for example, BeanDeserializer decodes from null. Same is true for CollectionDeserializer, array deserializers etc. Original idea was to reduce work for most deserializers (so like StringDeserializer need not consider this case in any way) but in the end not sure trade-off was correct one. But it is the design.

Given this, TupleDeserializer needs to handle nulls for fields and NOT delegate to JsonDeserializer. I can dig up examples if necessary; there's some complexity in providing "null replacement" values.