Schema support?

[zvrba]

One advantage of the built-in XmlSerializer is that we can use xsd.exe to generate an XML schema (which I use later to generate Java classes for interop between Java and C# worlds). Does ExtendedXmlSerializer support schema generation from the classes?

[Mike-E-angelo]

Hi @zvrba thanks for writing in with your question. ExtendedXmlSerializer is focused on CLR/.NET object graph serialization while attempting to preserve most of the classic XML functionality. The conceptual paradigm with this is that the C# class definition serves as the schema, eliminating the need for an xsd file or other schema mechanism altogether. This simplifies the overall design as well as reduce the number of artifacts required to successfully load and persist objects to/from memory.

If you are working with a XML/data from an external source that is outside of your control, my recommendation is to use XSLT which then formats/creates the XML that is necessary to serialize/deserialize objects within your application.

Hope that helps explain our approach here. If you have any further questions and/or require any further assistance around getting setup I will see if I can help. 👍

[zvrba]

I have full control over my C# classes, and my current workflow is: C# classes -> assembly/dll -> xsd.exe -> XSD schema -> xjc -> Java source -> Java classes. With the built-in XmlSerializer, this works like a charm and exchanging schematic data between C# and Java is truly easy. But I guess I got my answer: not supported.

Thanks for replying.

[Mike-E-angelo]

Ah indeed @zvrba this is more geared towards C#/.NET for sure at the moment. If I were to be interoperable with Java (or even Javascript 😁), I would use a service boundary and have the contracts be created/generated from there somehow.

This is good to think about perhaps heading into our v3 efforts but it is currently not supported in the workflow you expect. Thank you for sharing it, however. It is always much appreciated to get another viewpoint in this domain space.

Please do keep us in mind for any other XML-based serialization needs and don't be afraid to send another question. Hopefully we can help out next time. :) Closing this for now.

[Mike-E-angelo]

Actually... I've thought about this some more @zvrba and your scenario should still work as before, unless I am missing something very obvious. The xsd.exe uses attributes from the System.Xml namespace but it doesn't use the XmlSerializer to produce its artifacts. That is, I believe the schema that is produced is via a XmlDocument not the XmlSerializer.

You should be able to still apply all the same attributes to your POCOs to produce the schema, and ExtendedXmlSerializer should just work with them (or ignore them if we don't have support for them yet).

I am going to reopen this and try out a quick test when I get some time to verify.

[zvrba]

Hi,

If I were to be interoperable with Java (or even Javascript 😁), I would use a service boundary and have the contracts be created/generated from there somehow.

Yes, that "somehow" being xsd.exe and xjc :) Do you have any other suggestions?

(Note that I must have data in XML as well because I put the documents into SQLServer XML columns and query/update them later through SQLServer's built-in support for XML processing. Plus, schemas can be attached to XML columns which allows the server to optimize the queries.)

You should be able to still apply all the same attributes to your POCOs to produce the schema

You're right, it should work, but the main reason I'm looking for alternative serializers is wider type support.

First: xsd.exe barfs on anything implementing IDictionary; it outputs an error message saying it doesn't implement this. Probably not so strange given that XML schema doesn't define dictionaries either. But still it'd be nice if they could be auto-serialized.

Second: the standard XmlSerializer barfs on nullable types. [XmlAttribute] Guid Id {get; set;} is serialized fine (xsd even outputs a Guid type). On the other hand, XmlSerializer barfs on Guid? Id {get; set;} even though xsd is able to output an optional element in the schema. It's possible to work around this using a private field and som logic in the property getters/setters, but it's.. verbose boilerplate code that's easy to get wrong.

[Mike-E-angelo]

Yes, that "somehow" being xsd.exe and xjc :) Do you have any other suggestions?

I am assuming you are dealing with two application boundaries. You could place the master/source (.NET) boundary behind an ASP.NET Core Service and customize the formatting to send to the Java client:

https://docs.microsoft.com/en-us/aspnet/core/web-api/advanced/formatting?view=aspnetcore-2.1#configuring-formatters

That is, you would return the objects that you need to consume in your Java-based service and have them serialized (if necessary) by the serializer that you prefer, then this would send to the Java client.

This is usually done by JSON, however. ASP.NET Core API uses Swagger, which can be used to generate Java-based client proxy code:

https://github.com/swagger-api/swagger-codegen

That is the direction I would take. This takes care of the Java client (or should), but that doesn't address this:

Note that I must have data in XML ...

Are you saying you generate the xsd and then use that to store data in these SQLServer XML columns? Or are you saying you store the actual serialized objects in XML columns?

If it's xsd's that you are storing, that is a wrinkle. If it's object graph data, that makes it easier as (mentioned earlier) ASP.NET Core API allows you to customize your output. I will need more context here to properly assist.

[zvrba]

Ok, more context: both C# and Java applications are desktop applications -- C# "backend" and a Java UI; there are no web services in the picture. They communicate asynchronously and bidirectionally using AMQP broker embedded on the Java side. (Bidirectional, session-based flow is not trivial to achieve with web services.) Each component in the C# app has its own AMQP session so the UI knows from which source the message is coming and can present it to the user appropriately. Objects are exchanged using AMQP messages containing XML serialized data.

Are you saying you generate the xsd and then use that to store data in these SQLServer XML columns? Or are you saying you store the actual serialized objects in XML columns?

Currently I'm only storing serialized objects in XML columns. In the future, I'll add XSDs to column definitions to increase performance (SQLServer uses XSD to "shred" XML documents into corresponding native data types and indexed by paths to elements/attributes and thus speeds up queries.).

PS: On the C# side, the library (AMQPNetLite) has capabilities for own contract-based serialization of classes to standard AMQP "frames". Nothing similar exists on the Java side, where I'd get "reduced" to key-value dictionary lookups. So, XML is the only common denominator.

[Mike-E-angelo]

OK I think I am onboard with your scenario. Thank you for taking the time to explain it.

Bidirectional, session-based flow is not trivial to achieve with web services

No doubt. Now that I have a better understanding, the intention with the ASP.NET Core API magic is simply to produce your Java proxy classes. Everything else would remain the same with the exception of perhaps using ExtendedXmlSerializer to serialize your objects into XML.

However, there is a new problem in such a case. :) If you are able to serialize objects such as IDictionary on .NET side, are you certain you will be able to deserialize them on the Java-side?

[zvrba]

Ha, that's a good point. I guess I have to stick with XmlSerializer. Thanks for your comments.

[Mike-E-angelo]

Yeah... bummer. Fun to think about, though. We've had a major disruption in this space, with WSDL/SOAP/WCF being the "standard" but then somehow got hijacked into the cluster we have today. 😅 I have yet to build anything to use any of the new API magic, but I do intend to do so soon try to see how it can work with different scenarios when possible.

Alright, closing this again. Feel free to contribute more if you have further comments/questions.

[zvrba]

Actually, now I'm looking with fresh mind on your suggestion about using ASP.NET Core magic... I will eventually need XSD for SQLServer. But you're right, I don't need to have XSD if I can generate Java proxies in another way! :)

Can you explain me how would I coerce ASP.NET Core / Swagger into generating Java proxies based on XML attributes in C# ([XmlType], [XmlRoot] et cetera).

[Mike-E-angelo]

That's the spirit! 😆

So the idea here is that you are really leveraging the "JSON" serialization model, which is more robust than XML's and should get you all of your properties as you have them modeled out in C#.

The bad news on my end is that I have not yet to do this. The idea is that you would:

• Create an ASP.NET Core API Project that references your C# model.
• Launch/host this API. The API contains and produces the the Swagger definition containing your classes/model upon request.
• Use something like this code generator to load the Swagger definition and generate your Java classes.

With a little luck, the Java classes should look like the C# classes. I will see if I can find some additional resources for you in regards to this. Opening this up to keep the discussion going. 🎉

[zvrba]

Thanks for your comments!

The very first point is the stumbling block: will ASP.NET Core understand XML annotations when building its model? And I'm less than happy for having to use crappy JSON due to it having only three primitive types (string, number, null). XSD/XML is 100 times more expressive, which is why I'm using it in the first place. :) (I have DateTime fields as well as Guids that XmlSerializer understands. xsd.exe even outputs a XSD definition of GUID with correct regexp; though Java's xjc maps it to a plain string.)

[Mike-E-angelo]

And I'm less than happy for having to use crappy JSON

You don't have to sell me... I wrote a serializer AFTER the JSON hysteria struck. 😅

That said, you shouldn't have to deal with JSON at all. If I understand correctly, the goal is simply to get your Java classes generated whenever you change your schema in C#, correct?

From what I understand, OpenAPI/Swagger will look at a C#/model and build the schema from that and serve it from a definition from ASP.NET Core API. You can then point a code generator (that understands JSON) to that definition and it will generate the code from that.

So, if all goes well, you shouldn't need to use anything from System.Xml or attributes at all. Assuming the names and symbols line up correctly.

HOWEVER :) If you are using attributes (that is, formatting your data into XML attributes and using CLR/C# metadata to define/describe them) and such, that will be a problem.

[Mike-E-angelo]

Ooooh... looks like they do have some support for XML?

https://swagger.io/docs/specification/data-models/representing-xml/

This will take some investigation... learning this along with you. :)

[Mike-E-angelo]

Here's some actual code, pretty much what I was thinking of... you pretty much just have to hold your nose around the JSON and keep in mind the end goal of generating your proxy classes:

https://docs.microsoft.com/en-us/aspnet/core/tutorials/web-api-help-pages-using-swagger?view=aspnetcore-2.1

[zvrba]

If you are using attributes (that is, formatting your data into XML attributes and using CLR/C# metadata to define/describe them) and such, that will be a problem.

Yes, that's what I use. [XmlElement], [XmlAttribute] et.al. from System.Xml.Serialization namespace. The data model has grown rather big and it's already being persisted to the database, so it's a no-go to convert it into something else.

---

Originally, my plan was to use Microsoft Bond to describe the data model. (Bond also uses C# attributes to describe serialization.) However, converting it to XSD, even programmatically through reflection, has proven itself to be challenging and I just reverted to using XSD for everything...

[Mike-E-angelo]

Bummer... yeah this is a tricky problem. I myself absolutely despise the disparity between naming conventions, e.g. propertyName vs. PropertyName. So you will have all sorts of metadata around your classes to compensate, thereby polluting your classes, etc.

So for the v3 serializer, I am attempting to do something similar with HTML output. You describe your UI elements in XAML-esque components (DockPanel, Grid, etc.) and these elements then get generated into their HTML counterparts.

The same could be done with XSD? Essentially what is occurring is a mapping between objects in memory to an output in text. Essentially an object-view generation. Sounds simple until you actually have to implement it. :P

I will keep this in mind as I go forward. There are actually 3 scenarios now:

• Serialization (default)
• HTML Generation
• Schema (XSD) Generation

The idea is that you would create a view generator provider and have it work its magic based on the object it is processing, so in your case an XsdViewGenerator, I am thinking.

[zvrba]

I have dug more into XmlSerializer and found two workarounds for "difficult" types like dictionaries: IXmlSerializable and https://docs.microsoft.com/en-us/dotnet/api/system.xml.serialization.xmlschemaproviderattribute?view=netcore-2.1 The first allows to do serialization manually, while the other allows to define the schema of custom serialization that xsd.exe will use for generating schemas.

I just wish Microsoft had established a convention for serializing dictionaries that xsd.exe recognizes, like they have for Guid.

[Mike-E-angelo]

Cool... yeah that API has been there from the start and there were a lot of problems/domains that they were solving around that time. Glad to hear that you have found something for your scenario, and it's unfortunate that we don't have something in place to help out further. I'll keep this in mind though going forward.

Closing for now... feel free to contribute more to this thread if you find more information that you would like to share. I know I will. :)