[API Proposal]: Add a `JsonUnknownTypeHandling` setting for deserializing `object` to .NET primitive values

Background and motivation

Branching off from the conversation in #29960 and #97801 to consider a potential built-in object deserializer that targets .NET primitive values as opposed to targeting the DOM types: JsonNode or JsonElement. The background is enabling users migrating off of Json.NET needing a quick way to support object deserialization, provided that the deserialized object is "simple enough". This approach is known to create problems w.r.t. loss of fidelity when roundtripping, which is why it was explicitly ruled out when STJ was initially being designed. It is still something we might want to consider as an opt-in accelerator for users that do depend on that behaviour.

This proposal would map JSON to .NET types using the following recursive schema:

JSON null maps to .NET null.
JSON booleans map to .NET bool values.
JSON numbers map to int, long or double.
JSON strings map to .NET string values.
JSON arrays map to List<object?>.
JSON objects map to Dictionary<string, object?>.

Here's a reference implementation of the above:

public class NaturalObjectConverter : JsonConverter<object>
{
    public override object? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        => ReadObjectCore(ref reader);

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
    {
        Type runtimeType = value.GetType();
        if (runtimeType == typeof(object))
        {
            writer.WriteStartObject();
            writer.WriteEndObject();
        }
        else
        {
            JsonSerializer.Serialize(writer, value, runtimeType, options);
        }
    }

    private static object? ReadObjectCore(ref Utf8JsonReader reader)
    {
        switch (reader.TokenType)
        {
            case JsonTokenType.Null:
                return null;

            case JsonTokenType.False or JsonTokenType.True:
                return reader.GetBoolean();

            case JsonTokenType.Number:
                if (reader.TryGetInt32(out int intValue))
                {
                    return intValue;
                }
                if (reader.TryGetInt64(out long longValue))
                {
                    return longValue;
                }

                // TODO decimal handling?
                return reader.GetDouble();

            case JsonTokenType.String:
                return reader.GetString();

            case JsonTokenType.StartArray:
                var list = new List<object?>();
                while (reader.Read() && reader.TokenType != JsonTokenType.EndArray)
                {
                    object? element = ReadObjectCore(ref reader);
                    list.Add(element);
                }
                return list;

            case JsonTokenType.StartObject:
                var dict = new Dictionary<string, object?>();
                while (reader.Read() && reader.TokenType != JsonTokenType.EndObject)
                {
                    Debug.Assert(reader.TokenType is JsonTokenType.PropertyName);
                    string propertyName = reader.GetString()!;

                    if (!reader.Read()) throw new JsonException();
                    object? propertyValue = ReadObjectCore(ref reader);
                    dict[propertyName] = propertyValue;
                }
                return dict;

            default:
                throw new JsonException();
        }
    }
}

The reference implementation is intentionally simplistic and necessarily loses fidelity when it comes to its roundtripping abilities. A few noteworthy examples:

Values such as DateTimeOffset, TimeSpan and Guid are not roundtripped, instead users get back the string representation of these values. This is done intentionally for consistency, since such a deserialization scheme cannot support all possible types that serialize to string.
Non-standard numeric representations such as NaN, PositiveInfinity and NegativeInfinity currently serialized as strings using the opt-in JsonNumberHandling.AllowNamedFloatingPointLiterals flag are not roundtripped and are instead returned as strings.
Numeric values can lose fidelity (e.g. decimal.MaxValue gets fit into a double representation).

API Proposal

namespace System.Text.Json.Serialization;

public enum JsonUnknownTypeHandling
{
    JsonElement,
    JsonNode,
+    DotNetPrimitives,
}

API Usage

var options = new JsonSerializerOptions { UnknownTypeHandling = JsonUnknownTypeHandling.DotNetPrimitives };

var result = JsonSerializer.Deserialize<object>("""[null, 1, 3.14, true]""", options);
Console.WriteLine(result is List<object>); // True
foreach (object? value in (List<object>)result) Console.WriteLine(value?.GetType()); // null, int, double, bool

Alternative Designs

Do nothing, have users write their own custom converters.

Risks

There is no one way in which such a "natural" converter could be implemented and there also is no way in which the implementation could be extended by users. There is a good risk that users will not be able to use the feature because they require that the converter is able to roundtrip DateOnly or Uri instances, in which case they would still need to write a custom converter from scratch.

cc @stephentoub @bartonjs @tannergooding who might have thoughts on how primitives get roundtripped.

Author:	eiriktsarpalis
Assignees:	-
Labels:	`api-suggestion`, `area-System.Text.Json`, `untriaged`
Milestone:	-

dotnet / runtime