W# for interoperability

[FilippoPolo]

Hi all,

We have a fairly large and complex web application that's built with W#, and we have the following problem. We want to make requests to a remote server, which accepts websockets on which JSON objects are exchanged. These JSON objects are actually serialised .NET objects; because of this, we would really, really like to be able to just use W#-generated deserialisers on our side. This would save us a lot of time and make the whole thing a lot easier to maintain.

The problem is, the server uses a different serialisation format, especially for F# structures like tuples and options and whatnot; they all end up as JSON that doesn't look like the JSON that W# serialisers understand.

Currently, we deal with this with a handcrafted translation layer, but this negates most of the benefit of using generated deserialisers.

The server doesn't use W#, and they don't want to bring in the entire W# stack as a dependency, just for us. This is understandable, especially considering that if you import the W# NuGet package, I think that currently it makes the entire project build with W#; we can't ask them this.

If we could have the W# serialisers as a standalone library that doesn't have a ton of dependencies and doesn't modify the build process, however, I think we could get them to use it to give us a W#-serialised endpoint. That would be a perfect solution for us. I think it would be a pretty good thing in an abstract sense for W# too, because serialisation is closely related to interoperability, and a small self-contained library specifically for interop makes a lot of sense.

Is there any way to get the JSON serialisation bits of W#, without bringing in the whole stack?

[Tarmil]

especially considering that if you import the W# NuGet package, I think that currently it makes the entire project build with W#; we can't ask them this.

Referencing the WebSharper.FSharp or WebSharper.CSharp package and adding a WebSharper project configuration (ie <WebSharperProject> in .*proj or a wsconfig.json file) would make the project build with WebSharper; but to use the serialization, you only need to reference the WebSharper package, which is just a library.

That being said, I agree that it would be useful to be able to use the serialization without referencing the rest of the library.

[cgravill]

Might it be possible to provide optional compatibility with the new JSON serializer being added to .NET Core? https://devblogs.microsoft.com/dotnet/announcing-net-core-3-0-preview-5/#user-content-introducing-the-json-serializer-and-an-update-to-the-writer

[FilippoPolo]

Might it be possible to provide optional compatibility with the new JSON serializer being added to .NET Core? https://devblogs.microsoft.com/dotnet/announcing-net-core-3-0-preview-5/#user-content-introducing-the-json-serializer-and-an-update-to-the-writer

That would be pretty awesome. The new JSON serializer is probably going to become the standard for converting between .NET and JSON. If W# could talk to that, all of these issues would be solved.

[granicz]

Can you give some examples of what your server side expects for tuples and unions? The built-in WebSharper features might be sufficient to get to that output.

[Jand42]

Expanding a bit on @Tarmil's answer: referencing WebSharper.Core.dll only should be enough for using WebSharper.Json.Serialize/Deserialize. You can use exclude in paket.references for all other dlls or just add a direct reference. Or if you would like to go this way, we could also create a separate nuget package WebSharper.Core only.

[FilippoPolo]

Can you give some examples of what your server side expects for tuples and unions? The built-in WebSharper features might be sufficient to get to that output.

Example for a union: type t = UnionCase of int let instance = UnionCase 1 'instance' serialises to: {"Case":"UnionCase","Fields":[1]}

Example for a tuple: let instance = (1,2) 'instance' serialises to: {"Item1":1,"Item2":2}

Example for an option: let instance = Some 1 'instance' serialises to: {"Case":"Some","Fields":[1]}

This is Newtonsoft.Json, a fairly common JSON/.NET serialisation library. I don't think it's aware of F# constructs; it just serialises the underlying .NET representation.

[FilippoPolo]

Expanding a bit on @Tarmil's answer: referencing WebSharper.Core.dll only should be enough for using WebSharper.Json.Serialize/Deserialize. You can use exclude in paket.references for all other dlls or just add a direct reference. Or if you would like to go this way, we could also create a separate nuget package WebSharper.Core only.

That could be helpful. Could you give me an example of how to use WebSharper.Core to serialise/deserialise? I see there's several classes in WebSharper.Core.Json, but I'm not clear on how to use them to get a string from an object and vice-versa.

[Jand42]

@FilippoPolo The WebSharper.Core.Json module contains functions for parsing json format into an F# data representation (Json.Value), not arbitrary types. I was talking about WebSharper.Json which uses this internally then produces custom typed values with reflection.

WebSharper.Json serialization is customizable to a level with attributes, see documentation at https://developers.websharper.com/docs/v4.x/fs/json .

It is intended to be flexible enough to create a well-readable JSON API, that are always compatible between client and server WebSharper projects. But it does not support overriding deserialization/serialization rules yet. For example enums have their field values in separate named json fields, and tuples are serialized as JSON arrays, options are serialized as missing/existing fields if inside a record/union.

So if you have very specific deserialization needs, you can still fall back to WebSharper.Core.Json that you mentioned, Parse/Stringify are converting between a string and a Json.Value but then you need your own logic on top of this to process/produce the Json.Values.

[FilippoPolo]

Okay, then it's the same API I've used before. I thought you were telling me to use a different API, because there is no WebSharper.Json in WebSharper.Core.dll; it's in WebSharper.Web. It still requires Core, so I need to add two DLLs.

Thanks; I'll see if this can work for us. A separate NuGet package would definitely help, though.

[FilippoPolo]

Hi all,

I've got a working proof-of-concept that uses W# serialisation, and it allows me to vastly simplify message handling on the browser side. I can get rid of an entire layer this way!

The only problem is how to reference W# without bringing in the whole stack. We don't want to reference specific DLLs directly, because it's brittle compared to using NuGet packages. We've tried using exclude, but it doesn't seem to work (at least, not with paket, which is what we're using).

Could you provide a NuGet package that's oriented towards interop, and contains just WebSharper.Core and WebSharper.Web?

[Jand42]

@FilippoPolo The parts of the code contained in WS.Web.dll are a couple lines only, I can move this to WS.Core.dll only. Testing it soon.

[FilippoPolo]

That would be great, thanks! Could you also make it into a standalone NuGet package?

[Tarmil]

@Jand42 @granicz Do you think we should go one step further and isolate just the JSON part in a separate assembly and nuget? It could even be referenced by Bolero instead of including a reimplementation there.

[Jand42]

@FilippoPolo Yes, I am packaging it. @Tarmil Not a bad idea, but this would be contained in a non-WebSharper/Bolero namespace? Optimally this would be used by WS instead of duplication (a breaking change), but this is hard because of the attributes that can be used for other JS interop (non-JSON API) purposes too in WS (Constant, Name, OptionalField).

[Jand42]

@FilippoPolo The WebSharper.Core package is now available at https://daily.websharper.com/nuget I will need to do a full stack build and some testing to ensure moving this class do not cause any breaks down the line, so it will be released on nuget.org later.

[FilippoPolo]

Awesome, thanks! I'll need some time to integrate it on our side, then I'll let you know how it goes.

[FilippoPolo]

@Jand42 It seems to be working well for me. Could you publish WebSharper.Core on nuget.org so we can integrate it into our main branches?

[Jand42]

@FilippoPolo Thanks for the feedback! I am building a new version of the stack to release, to ensure binary compatibility.

[Jand42]

@FilippoPolo I am really sorry for the delay, some build issues and traveling has held me up. Now up on https://www.nuget.org/packages/websharper.core

[FilippoPolo]

We're now using this, and it works perfectly. Thanks again!

[nightroman]

Please take a look at the referenced new issue. Something is not quite right with the new serialization performance.

[Jand42]

@FilippoPolo Resolved there, there was a mistake in refactoring indeed, sorry. Releasing new packages right now.