DataTable serialization in SOAP Client

[sudoudaisuke]

I'm converting .net framework winform apps to .net core 3.0. My apps call many SOAP methods. SOAP web service server is implemented by ASP.NET webform based asmx(wdsl). SOAP method returns typed DataTable.

I know that .net core will not support ASP.NET webform. But, if SOAP client (generated by dotnet-svcutil) will not support it, then I cannnot convert client side winform apps to .net core and .NET 5. So please support client side datatable serialization.

3242

3378

[StephenBonikowsky]

According to https://referencesource.microsoft.com/#System.Data/System/Data/DataTable.cs,5906

Typed DataTable is not supported in WSDL (SQLBU 444636) That isn't something WCF can do anything about.

If #3378 will solve your problem @sudoudaisuke then that same bug is already tracking that work.

@dasetser Is there anything else you would like to add?

[StephenBonikowsky]

@sudoudaisuke Could you help me understand a few things that are confusing me.

I'm not sure what you mean by a "SOAP method", do you just mean the Server side operation and you are saying that it returns the DataTable type? Any WSDL generated for the Service wouldn't include that type since it isn't supported. So running dotnet-svcutil against that WSDL wouldn't accomplish anything.

If on the other hand you are just getting data from a service operation and on the client side you want to populate a DataTable type with that data you could do it by reading the data in the SOAP message into some other object and then populating a DataTable with it.

[StephenBonikowsky]

No other action we can take on this, please let us know if there is any additional info you can provide or anything else we can help with.

[sudoudaisuke]

Hi @StephenBonikowsky . Thanky for your reply. And sorry for my late reply.

This is my sample Visual Studio solution files, and wsdl, xsd.

TypedDatatTableAsmxSolution.zip

wsdl_and_xsd.zip

If .NET Framework app can generate TypedDataSet from wsdl, but .NET Core app cannot.

I can do like this.

1. Define TypedDataSet in .NET Standard 2.0 lib.
2. Reference the lib, from server (Framework / webservice / asmx) and client (.NET Core).
3. In client, call generated ClientBase method (return XElement not TypedDataTable).
4. In client, create TypedDataTable Instance.
5. In client, call DataTable.ReadeXml method.

var client = new ServiceReference1.WebService1SoapClient(ServiceReference1.WebService1SoapClient.EndpointConfiguration.WebService1Soap);
var data = await client.GetSampleDataTableAsync();

var table = new TypedDataTableDefinition.TypedSampleDataSet.SampleDataTableDataTable();

using (var stream = new MemoryStream()) {
    data.Any1.Save(stream);
    stream.Seek(0, SeekOrigin.Begin);
    table.ReadXml(stream);
}

But, I have many TypedDataSets and many asmx mehods. .NET Core takes time and effort. .NET Framework is better for me. I want that .NET Core can generate TypedDataSet from wsdl and xsd too.

[StephenBonikowsky]

@sudoudaisuke Thanks for the additional info, we'll take a look at this.

[StephenBonikowsky]

@imcarolwang In the additional info provided by @sudoudaisuke he says...

If .NET Framework app can generate TypedDataSet from wsdl, but .NET Core app cannot.

Could you verify using the project and info provided that this works on full framework and fails on .NET Core?

[imcarolwang]

I've confirmed the behavior reported in issue that when generating wcf proxy code, a DataSet/DataTable type could be deserialized as its original type in .NETFramework wcf client app using svcutil.exe, but would be deserialized to other alternatives like System.Xml.Linq.XElement[] in .NETCore with dotnet-svcutil.exe.

I found Scott Hanselman's document here stating that return a DataSet in web service is not a good idea, and there's discussion about the topic on stackoverflow. @StephenBonikowsky base on the information probably the behavior difference is by design when developing features on Core? Need feature owner's further confirm.

[StephenBonikowsky]

Thanks @imcarolwang @dasetser @mconnew Is this a purposeful change between svcutil.exe and dotnet-svcutil.exe?

[dasetser]

This is a purposeful change between svcutil.exe and dotnet-svcutil.exe that was made because dotnet-svcutil was written to support .NET Core 1.0 apps, and these System.Data APIs were not available in 1.0. Now that 1.0 is out of support I believe we could look into supporting these, but I don't think we have time to investigate it for the current release.

[dasetser]

We had another report of this issue on #3932. As part of investigating that developer community issue I looked into this a little more, and it's not as easy as I was thinking it would be. While these System.Data APIs now exist on .NET Core, the way we supported these in wsdl does not. This is still a future work item, but I'll add some notes on it based on my investigation to help when we come back to this.

Essentially what happens is the wsdl declares this as an xs:any element, which could be anything, then puts an annotation in that this is actually a DataSet or DataTable. That's why dotnet-svcutil generates this as a generic type. In the .NET Framework the System.Data code has a custom importer that includes reading this in and correcting the type, and svcutil normally adds that importer so that it works. Unfortunately these importers did not get ported to .NET Core. In order to support this on dotnet-svcutil we would need to port these ourselves, or rewrite a similar importer to detect the annotation and handle it correctly.

[EamonHetherton]

I'm hitting this too. Trying to port some .NET 4.5 code to core 3.1. This code depends on a webservice originally written in .NET 2 that returns typed datasets.

If I create a .NET framework library and "Add Service Reference" it correctly generates the service proxy, but in .NET core using "Add Connected Service" -> "Microsoft WCF Web Service Reference Provider" the service proxy returns System.Xml.XmlElement instead of a dataset. Worse still, this XElement is only the schema part of the result so I can't even manually turn it into a dataset.

Interestingly though, if I replace System.Xml.XmlElement with System.Data.DataSet in the generated code it works. I can live with an untyped dataset, can't use an XmlElement as it is.

I've just done a find->replace of "System.Xml.XmlElement" with "System.Data.DataSet" in Reference.cs for now as this service is simple and all the result types are DataSets and removed the "inputs" section from ConnectedService.json so not one accidently regenerates this.

[jcbvm]

I encountered the same problem a while ago. I'm using the following workaround currently to transform XElements into a DataSet:

private DataSet ToDataSet(ArrayOfXElement data)
{
  DataSet result = new DataSet();
  string rawXml = new XElement("Root", data.Nodes).ToString();
  using (StringReader reader = new StringReader(rawXml))
  {
    result.ReadXml(reader);
  }
  return result;
}

[Peperud]

Just got to this myself, while converting an old project. In my case it is a DataTable being passed as an input parameter.

[Durgagithu]

I am facing similar issue where I need to call webservice which sends and returns Dataset. How can this be resolved?

[eaydemir]

I encountered the same problem a while ago. I'm using the following workaround currently to transform XElements into a DataSet:

private DataSet ToDataSet(ArrayOfXElement data)
{
  DataSet result = new DataSet();
  string rawXml = new XElement("Root", data.Nodes).ToString();
  using (StringReader reader = new StringReader(rawXml))
  {
    result.ReadXml(reader);
  }
  return result;
}

how can we do the opposite? I mean DataSet to ArrayOfXElement.

[Durgagithu]

I encountered the same problem a while ago. I'm using the following workaround currently to transform XElements into a DataSet:

private DataSet ToDataSet(ArrayOfXElement data)
{
  DataSet result = new DataSet();
  string rawXml = new XElement("Root", data.Nodes).ToString();
  using (StringReader reader = new StringReader(rawXml))
  {
    result.ReadXml(reader);
  }
  return result;
}

how can we do the opposite? I mean DataSet to ArrayOfXElement.

I encountered the same problem a while ago. I'm using the following workaround currently to transform XElements into a DataSet:

private DataSet ToDataSet(ArrayOfXElement data)
{
  DataSet result = new DataSet();
  string rawXml = new XElement("Root", data.Nodes).ToString();
  using (StringReader reader = new StringReader(rawXml))
  {
    result.ReadXml(reader);
  }
  return result;
}

how can we do the opposite? I mean DataSet to ArrayOfXElement.

Any luck with this issue?

[PimVendrig]

To convert from DataSet to ArrayOfXElement I am using the following method:

public ArrayOfXElement DataSetToArrayOfXElement(DataSet dataSet)
{
    if (dataSet == null) throw new ArgumentNullException(nameof(dataSet));

    var arrayOfXElement = new ArrayOfXElement();

    using (var schemaStream = new MemoryStream())
    {
        using (var writer = XmlWriter.Create(schemaStream))
        {
            dataSet.WriteXmlSchema(writer);
        }
        schemaStream.Position = 0;
        arrayOfXElement.Nodes.Add(XElement.Load(schemaStream));
    }

    using (var dataStream = new MemoryStream())
    {
        using (var writer = XmlWriter.Create(dataStream))
        {
            dataSet.WriteXml(writer, XmlWriteMode.DiffGram);
        }
        dataStream.Position = 0;
        arrayOfXElement.Nodes.Add(XElement.Load(dataStream));
    }

    return arrayOfXElement;
}

And to convert from ArrayOfXElement to DataSet, I am using:

public DataSet ArrayOfXElementToDataSet(ArrayOfXElement arrayOfXElement)
{
    if (arrayOfXElement == null) throw new ArgumentNullException(nameof(arrayOfXElement));
    if (arrayOfXElement.Nodes.Count != 2) throw new ArgumentException($"{nameof(arrayOfXElement)} should have 2 {nameof(arrayOfXElement.Nodes)}.", nameof(arrayOfXElement));

    var dataSet = new DataSet();

    using (var schemaStream = new MemoryStream())
    {
        using (var writer = XmlWriter.Create(schemaStream))
        {
            arrayOfXElement.Nodes[0].WriteTo(writer);
        }
        schemaStream.Position = 0;
        dataSet.ReadXmlSchema(schemaStream);
    }

    using (var dataStream = new MemoryStream())
    {
        using (var writer = XmlWriter.Create(dataStream))
        {
            arrayOfXElement.Nodes[1].WriteTo(writer);
        }
        dataStream.Position = 0;
        dataSet.ReadXml(dataStream, XmlReadMode.DiffGram);
    }

    return dataSet;
}

[fededim]

I am experiencing the same issue of @sudoudaisuke. The problem is that in the autogenerated code of dotnet-svcutil the XSD referenced by the WSDL with import tag

<s:import schemaLocation="http://xx.yy.zz/ReticaServer/ReticaSara.asmx?schema=EvToAdf" namespace="http://tempuri.org/EvToAdf.xsd"/>
<s:import schemaLocation="http://xx.yy.zz/ReticaServer/ReticaSara.asmx?schema=DefEvento" namespace="http://tempuri.org/DefEvento.xsd"/>
<s:import schemaLocation="http://xx.yy.zz/ReticaServer/ReticaSara.asmx?schema=GestEventi" namespace="http://tempuri.org/EventiInviati.xsd"/>

are not transformed into classes, but they are instead mapped to a generic class composed by 2 properties "XmlElement Any1" and "XmlElement[] Any" which are quite obscure, e.g. I did not find any way to use them (except the sudoudaisuke's suggested workaround). It is a different behaviour from standard .NET svcutil and I would like to know the reasons of this different behaviour (hasn't it been implemented yet ?)

Federico

    public partial class insertAttributiDtInp
    {

        private System.Xml.XmlElement[] anyField;

        private System.Xml.XmlElement any1Field;

        private string namespaceField;

        private string tableTypeNameField;

        public insertAttributiDtInp()
        {
            this.namespaceField = "http://tempuri.org/DefEvento.xsd";
            this.tableTypeNameField = "ATTRIBUTI_SARADataTable";
        }

        /// <remarks/>
        [System.Xml.Serialization.XmlAnyElementAttribute(Namespace="http://www.w3.org/2001/XMLSchema", Order=0)]
        public System.Xml.XmlElement[] Any
        {
            get
            {
                return this.anyField;
            }
            set
            {
                this.anyField = value;
            }
        }

        /// <remarks/>
        [System.Xml.Serialization.XmlAnyElementAttribute(Namespace="urn:schemas-microsoft-com:xml-diffgram-v1", Order=1)]
        public System.Xml.XmlElement Any1
        {
            get
            {
                return this.any1Field;
            }
            set
            {
                this.any1Field = value;
            }
        }

        /// <remarks/>
        [System.Xml.Serialization.XmlAttributeAttribute()]
        public string @namespace
        {
            get
            {
                return this.namespaceField;
            }
            set
            {
                this.namespaceField = value;
            }
        }

        /// <remarks/>
        [System.Xml.Serialization.XmlAttributeAttribute()]
        public string tableTypeName
        {
            get
            {
                return this.tableTypeNameField;
            }
            set
            {
                this.tableTypeNameField = value;
            }
        }
    }

[chucker]

@fededim it's flat-out not implemented in .NET Core. Your best bet is to manually run .NET Framework's version of svcutil instead.

[OronDF343]

Here's my workaround for converting Any and Any1 to DataTable based on the solution from @PimVendrig but with some tweaks. See comment above by @fededim for an example of when to use this. I have not tested reading to DataSet but you should be able to just replace DataTable with DataSet in the code.

    public static DataTable ToDataTable(XmlElement[] any_schema, XmlElement any1_diffGram)
    {
        var dataTable = new DataTable();

        using (var stream = new MemoryStream())
        {
            using (var writer = XmlWriter.Create(stream))
            {
                writer.WriteStartElement("root");

                foreach (var item in any_schema)
                    item.WriteTo(writer);
                any1_diffGram.WriteTo(writer);

                writer.WriteEndElement();
            }
            stream.Position = 0;
            dataTable.ReadXml(stream);
        }

        return dataTable;
    }

[moslemkh]

public static DataTable ConvertToDataTable(this XmlElement data) { StringReader theReader = new StringReader(data.InnerXml); DataSet theDataSet = new DataSet(); theDataSet.ReadXml(theReader); return theDataSet.Tables[0];
}

[nandu14anand]

@dasetser @HongGit @StephenBonikowsky @mconnew

waiting for updates on this issue, as I am also trying to change my client side from windows Application to .Net core web API and the reference.cs is missing to have the specific/paticular Dto/Datatable/Dataset as type.

provide some update, is it fixed already?

[mconnew]

Here's the history of the problem. Originally the class XmlSchema wasn't supported in .NET Core. The class existed, but it had no members and was there as a placeholder so as not to break classes api's which referenced it. The specific api that was needed to be implemented for dotnet-svcutil to work was IXmlSerializable.GetSchema. While that api existed, there wasn't a real implementation of XmlSchema that could be populated.
XmlSchema was implemented in .NET Core 2.0, but we had the problem of still needing to run on .NET Core 1.0. Since then (quite a while ago now), all supported versions of .NET have had XmlSchema available. But we have another problem now.
When dotnet-svcutil was created, much of the needed dependencies in the rest of .NET [Core] needed for dotnet-svcutil to do it's job were absent. This resulted in needing to embed a private fork of large chunks of .NET Framework into dotnet-svcutil itself. The result of this is that there are similarly/identically named types internal to dotnet-svcutil which are technically different types to the .NET runtime. When dotnet-svcutil sees the annotation saying the real type is DataSet, it can't instantiate an instances and cast it to IXmlSerializable because the version of IXmlSerializable known to dotnet-svcutil is a different runtime type to what's in the .NET runtime, so the case fails and it looks like it doesn't implement it.

The work that needs to be done is to remove the FrameworkFork from dotnet-svcutil and use everything from .NET. The problem we then have is there are some WCF api's missing to be able to import the bindings, specifically the interfaces IWsdlImportExtension and IPolicyImportExtension, along with the implementations to import the bindings. The plan is to add these interfaces to WCF along with the supporting classes like WsdlImporter and MetadataImporter and have each of the bindings/binding elements provide their own implementation. Eventually dotnet-svcutil will be a small utility which just plumbs everything together from the WCF libraries and the .NET runtime with minimal implementation itself. This is going to take time as we don't have anyone dedicated to this full time.

[nandu14anand]

Thanks for the update @mconnew .

But whats the work around then? Its really difficult to work with ArrayOfXElement type, it has nodes, from which I can still fetch the data(response from WCF webservice), when it a simple get call, but it doesnt have any similarities with DTO/Dataset/Datatable.

scenario: The WCF webservice is using DTO.. and the ..asmx.cs file has endpoints now with parameter as Dto.. Added web reference for Soap WCF Webservice to a .Net Core Web API project. Now, 2 files are generated one is json and one is reference.cs. Now, here in reference.cs the types are totally changed.

If its just a get call, where no data should be filled from client side(web API), then I am some how able to fetch the response from the nodes and then structue the data back in same dto/dataset/datatable form. But the issue is, when I have to enter some data in dataset or datatable on client side, before making the call, which will act as some kind of condition, on basis of which further processing and the data will be fetched. Here, in this case, what do you suggest I should proceed with.

If you are suggesting to modify reference.cs by some how not using ArrayOfXElement, Please provide some steps to do it. If not, please provide your suggestions. Eagerly waiting for your response. your inputs can be of help.

Regards

[mconnew]

I believe you can just edit the generated contract and replace ArrayOfXElement with the DataSet/DataTable etc type you are expecting. You could try using the .NET Framework implementation of svcutil with the /dataContractOnly parameter and then replace the generated data contracts with the ones from there. If you need to automate this, you could use the .NET Framework svcutil to create the data contract types only, then compile them into your app/library. Then when you use the dotnet-svcutil implementation, tell it to reuse existing types. As the types with these properties should match in name/namespace, I would expect this approach to work.