How to upgrade from 0.17.4 for single case unions and skippable fields?

[AlexGagne]

Hello, we were using the 0.17.4 version of the library and would like to upgrade it to the latest version of this library. However, we have a lot of serialized documents stored that can't be deserialized using the same options that we were using in 0.17.4.

Are we missing some configuration to deserialize as we did in 0.17.4? Or is this an instance of a bug?

Here's an example input that we cannot deserialize:

type SingleCaseUnion = SingleCaseUnion of string

type ToDeserialize =
    {
        SingleCaseUnion : SingleCaseUnion
        SkippableMember: int option
    }

let exampleBody = """
    {
        "singleCaseUnion": {
            "Case": "SingleCaseUnion",
            "Fields": [
                "some value"
            ]
        }
    }"""

Here are the options that we used back then:

module Serializer =
    let options =
        let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
        let jsonUnionEncodings = JsonUnionEncoding.UnwrapFieldlessTags ||| JsonUnionEncoding.UnwrapOption
        opt.Converters.Add(JsonFSharpConverter(jsonUnionEncodings, unionTagCaseInsensitive = true))
        opt
    let deserialize<'a> (body : string) =
        JsonSerializer.Deserialize<'a> (body, options)

I see that SkippableOptionsField was added and is now required for the skippable member. If I'm not mistaken, We can use UnionAdjacentTag to deserialize the single case union.

I also see that we have to switch to the Fluent JsonFSharpOptions to be able to use SkippableOptionsField. So I tried with the following configuration:

module Serializer =
    let options =
        let opt =
            JsonFSharpOptions()
                .WithUnwrapOption()
                .WithUnionUnwrapFieldlessTags()
                .WithUnionTagCaseInsensitive()
                .WithAllowNullFields()
                .WithSkippableOptionFields()
                .WithUnionAdjacentTag()
                .ToJsonSerializerOptions()
        opt.Converters.Add(JsonFSharpConverter())
        opt

    let deserialize<'a> (body : string) =
        JsonSerializer.Deserialize<'a> (body, options)

However, we get this error:

System.Text.Json.JsonException: Missing field for record type Playground.Types+ToDeserialize: SingleCaseUnion
   at System.Text.Json.Serialization.Helpers.failf@12.Invoke(String x)
   at System.Text.Json.Serialization.JsonRecordConverter`1.ReadRestOfObject(Utf8JsonReader& reader, Boolean skipFirstRead)
   at System.Text.Json.Serialization.JsonRecordConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.Serialization.JsonListConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Playground.Serializer.deserialize[a](String body) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Deserializer.fs:line 36
   at Playground.Program.main(String[] _arg1) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Program.fs:line 72

As if the AdjacentTag was being ignored? Or am I misunderstanding something?

[Tarmil]

I think it's failing to parse the record, rather than the union. The difference is the JsonSerializerDefaults.Web you used in the old version, which includes case-insensitive property names. You need it in the new version too. It's only usable with the constructor for JsonSerializerOptions, so instead of ToJsonSerializerOptions, you can use AddToJsonSerializerOptions:

let options =
    let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
    JsonFSharpOptions()
        (* all your options here... *)
        .AddToJsonSerializerOptions(opt)
    opt

You also don't need the extra opt.Converters.Add(JsonFSharpConverter()).

[AlexGagne]

Hello, thanks for your quick response. I think you are correct that the web defaults were missing. However, I still can't seem to deserialize my example.

type SingleCaseUnion = SingleCaseUnion of string

type ToDeserialize =
    {
        SingleCaseUnion : SingleCaseUnion
        SkippableMember: int option
    }

let exampleBody = """{"singleCaseUnion":{"Case":"SingleCaseUnion","Fields":["some value"]}}"""

let options =
        let opt = JsonSerializerOptions(JsonSerializerDefaults.Web)
        JsonFSharpOptions()
            .WithUnwrapOption()
            .WithUnionUnwrapFieldlessTags()
            .WithUnionTagCaseInsensitive()
            .WithAllowNullFields()
            .WithSkippableOptionFields(SkippableOptionFields.Always, true)
            .WithUnionAdjacentTag()
            .AddToJsonSerializerOptions(opt)
        opt

let deserialize<'a> (body : string) =
        JsonSerializer.Deserialize<'a> (body, options) 

I then get this exception:

System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'StartObject' as a string.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedString(JsonTokenType tokenType)
   at System.Text.Json.Utf8JsonReader.GetString()
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.Serialization.JsonUnwrappedUnionConverter`2.Read(Utf8JsonReader& reader, Type _typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.Serialization.Helpers.FieldHelper.Deserialize(Utf8JsonReader& reader)
   at System.Text.Json.Serialization.JsonRecordConverter`1.ReadRestOfObject(Utf8JsonReader& reader, Boolean skipFirstRead)
   at System.Text.Json.Serialization.JsonRecordConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Playground.Serializer.deserialize[a](String body) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Deserializer.fs:line 35
   at Playground.Program.main(String[] _arg1) in C:\Users\AlexGagné\RiderProjects\Playground\Playground\Program.fs:line 64

[AlexGagne]

Hello @Tarmil

Could you please confirm if this is a bug from the library or if I'm doing something wrong again?

I've tried different combination of options, removing the skippable member, trying different options, but couldn't manage to deserialize the body. I've also tried renaming all of my properties to UpperCamelCase and removing the Web defaults to see if it made a difference, but I got the exact same error (The The JSON value could not be converted to System.String error).

I cannot find a combination that deserializes my example body properly.

Serializing using my options, I see this being serialized: {"singleCaseUnion":"some value"}

I now see that the Adjacent Tag says that if there are no fields, then the fields property is excluded. I'm guessing this might be a breaking change that was done? If that is the case, is there a plan to re-enable this behaviour in the future? Or would we have to migrate all of our serialized data to follow this new rule?