Using Schema.NET in ASP.NET Core controllers in model binding

[veikkoeeva]

Hi!

I was wondering if you would have advice on how to use this interesting library in ASP.NET Core controllers so that roughly the following would work:

[HttpPost("[action]")]
public IActionResult Test([FromBody] Thing thing)
{
    var e = (Person)thing;
    return Accepted(new Uri("https://wwww.WhereToAskForResults.com"));
}

That is, accept JSON-LD data via a base classes. Or put other way, if I have a JSON document, say like at https://jsonld.com/person/, how could it be deserialized to a .NET object of Person.

[RehanSaeed]

I had a similar conversation with someone over Twitter a couple of days ago. The library only supports serialization from objects to JSON-LD at the moment and not deserialization from JSON-LD to object.

What's your use case? I think it should be fairly simple to add deserialization support. The Tweeter said he would try to add support and maybe submit a PR.

[veikkoeeva]

@RehanSaeed That would be a very welcomed addition. My use case is actually one you see in the code snippet, a web service that ingests JSON-LD data of various sorts, but thinking further, there are at least two parts to it:

1. Have a library that can read plain-text JSON-LD documents and transform them to strongly typed classes.
2. Have middleware or appropriate attributes to place that to ASP.NET Core pipeline.

I personally have the need only for the compacted form.

As the format is self-describing, it makes sense to be able to ingest it via base classes and then do pattern matching to the actual type. This brings one design consideration: Maybe it makes sense that if one doesn't have the most derived types present, some less derived class in the hierarchy could be used while the "excess" would be available somewhow for manipulation. The reason for this is that JSON-LD is a self-describing format so one could expect to receive all kinds of data and maybe not (yet) have the concrete, most derived types loaded into process at that process and/or point in time. Though this is more like a feature consideration now, maybe useful for discussion.

Taking a peek at the Twitter conversation, I took a look at the code too. I had this exactly same idea that the @type is the same as class together with the context being more like a namespacey thingy and those could be used as a discriminator. It would be very useful if the tooling you've developed already could be opened and adapted to ingest other than schema.org data to generate strong types (and make them have IEquatable, IStructuralEquatable etc. if possible).

Spelunking a bit through Internet, it looks there are efforts to have this functionality already, I don't know how useable they are. Some examples:

• https://github.com/wikibus/JsonLD.Entities
• https://github.com/NuGet/json-ld.net/issues/3
• https://github.com/digitalbazaar/jsonld.js/issues/157

I don't know if these libraries are functional in this ASP.NET Core example case, but it would seem there's a need for a library like this, and growing, and it would make sense to pool efforts to some library.

[veikkoeeva]

@RehanSaeed To add from the Twitter conversation, one use case about using the data in and out from the API has been something similar to http://adaptivecards.io/ and thinking this is actually the reason I found that project. As it looks like @matthidinger is one of the people behind that platform, this use case could be seen to tie to that too. I was thinking to use https://github.com/aurelia too and web components, a bit like at https://github.com/NuGet/json-ld.net/issues/3#issuecomment-240798216, but as noted, those are more like contemplating things after getting data in and out well enough, in a way or another. :)

[veikkoeeva]

One article that might give useful idea to deserialization: http://skrift.io/articles/archive/bulletproof-interface-deserialization-in-jsonnet/ or alternatively without attributes as in https://stackoverflow.com/questions/36969500/json-net-custom-jsonconverter-with-data-types.

[veikkoeeva]

Prototyping some code that might give a direction...

public void ConfigureServices(IServiceCollection services)
        {            
            services.AddMvc().AddJsonOptions(jsonOptions =>
            {
                Func<Type, bool> isConvertible = type => typeof(Thing).IsAssignableFrom(type);
                Func<JsonReader, Type, object, JsonSerializer, object> reader = (jsonReader, objectType, existingValue, serializer) =>
                {
                    var jObject = JObject.Load(jsonReader);

                    if(jObject.TryGetValue(nameof(Thing.Id), out JToken id))
                    {
                        //TODO: Use reflection here with the ID...
                        var jsonLdClass = ...
                        return serializer.Populate(jsonReader, jsonLdClass);
                    }

                    return null;
                };

                jsonOptions.SerializerSettings.TypeNameHandling = Newtonsoft.Json.TypeNameHandling.Auto;
                jsonOptions.SerializerSettings.Converters.Add(new GeneralJsonReadConverter(reader, isConvertible));
            });
        }

public class GeneralJsonReadConverter: JsonConverter
    {
        private Func<JsonReader, Type, object, JsonSerializer, object> Reader { get; }

        private Func<Type, bool> IsConvertible { get; }

        public override bool CanWrite { get; } = false;

        public override bool CanRead { get; } = true;

        public GeneralJsonReadConverter(Func<JsonReader, Type, object, JsonSerializer, object> reader, Func<Type, bool> isConvertible)
        {
            Reader = reader ?? throw new ArgumentNullException(nameof(reader));
            IsConvertible = isConvertible ?? throw new ArgumentNullException(nameof(isConvertible));
        }

        public override bool CanConvert(Type objectType)
        {
            return IsConvertible(objectType);
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            if(reader.TokenType == JsonToken.Null)
            {
                return null;
            }

            return Reader(reader, objectType, existingValue, serializer);
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
    }

Maybe notable here is that this could be used to write JSON-LD too. And that Thing probably isn't the right base class when thinking JSON-LD in general, but maybe something like JsonLd that has, the mandatory parts every JSON-LD class needs.

[RehanSaeed]

Some of those GitHub projects you've listed look interesting. What is missing (unless I'm mistaken after my cursory glance at the projects), that I think this project solves is that a property in Schema.org can be a single item or a collection, it can also have multiple types. To get around that problem, I use the four Values<T1..T4> structs e.g. For Values<string, Address>, you can actuall pass it astring,List,string[],Address,List

orAddress[]`. Any serialization strategy would need to support this.

The only way I can think of to do this, as I mentioned on Twitter, is to peek at the @type property then run that through an interface to convert it to a System.Type.

// Can't think of a better name.
public interface IJsonLdTypeToDotnetTypeConverter
{
    Type Convert(string type);
}

public class DefaultJsonLdTypeToDotnetTypeConverter : IJsonLdTypeToDotnetTypeConverter
{
    private readonly Dictionary<string, Type> types;

    public DefaultJsonLdTypeToDotnetTypeConverter()
    {
        // TODO, init types from Schema.NET.
    }

    public Type Convert(string type) => this.types[type];
}

The default implementation above would just lookup the nearly 1000 types in this Schema.NET project. Then someone could override the default implementation if they have their own custom derived types that they want to serialize.

Then all we need to do is plug the above code into the JsonConverter's already in the project, which is the hard part. This would require using JObject to peek in the JSON-LD for a @type property, if none is found, I suppose you could default to Thing or throw an exception, not sure which. You'd also need to check for JArray vs JObject and initialize the Values<T> struct accordingly.

[RehanSaeed]

Just to follow up, there are two scenarios here. One where you don't know what .NET type to deserialize to and one where you do. My message above caters to the first scenario.

[veikkoeeva]

@RehanSaeed Yep. It would be useful to have a system wherein I could receive JSON-LD data and then, say, by looking at @type I could construct the actual class. To that I end I was thinking that JSON.NET mechanism at make it work so that have conceptually a switch-case (a map) from @type to handlers that can both load assemblies if needed and from there set the properties in classes based on the received "raw JSON". Also if the @type could not be mapped to anything, it would be nice to have a "last chance handler" that could, say, store the data to somewhere for later processing.

It apperars I lost the notifications from this issue, sorry for that.

[RehanSaeed]

FYI, a PR has been submitted to add support for this. I'll find some time to review it soon but feel free to leave your own comments https://github.com/RehanSaeed/Schema.NET/pull/4.

[RehanSaeed]

The PR https://github.com/RehanSaeed/Schema.NET/pull/4 has been merged. You can now deserialize JSON to Schema.NET POCO types. The latest version will be on MyGet (See front page for link) in a few minutes or you can get at the source.

It would be great to test if it is working in ASP.NET Core model binding. Would really appreciate a comment here from someone doing some testing before I push the release to NuGet.

[veikkoeeva]

@RehanSaeed Just to be more clear, is the link to the library https://www.nuget.org/packages/Schema.NET/?

[RehanSaeed]

Yes. I was hoping someone would confirm that it's working before I did a release. I'll do that now anyway.

[ewassef]

Did this ever make it? I would like to do the same thing (model bind from json to c# classes) and its failing for me

[Turnerj]

Yeah, the referenced PR is merged and the library supports deserialization. Depending on what version you're using of ASP.NET, it could be using Newtonsoft.Json or System.Text.Json - currently the library supports the former and when .NET 6 lands, we will be able to instead support the latter.