Add support for ServiceMetadataBehavior features

[mconnew]

This includes support for:

• ?SingleWsdl and ?Wsdl pages to retrieve the wsdl
• html helper page which includes a link to the wsdl as well as client usage examples. This is used as a service health page
• mex endpoint supporting the IMetadataExchange contract on multiple bindings

[mconnew]

Some notes about implementation.
The html helper page needs to be identical to what WCF produces as some people have built tooling around the output from that page.
This has a dependency on types in the System.Web.Services.Description. I have opened an issue dotnet/wcf#4464 in the wcf repo to track creating a package holding these types.

[StephenBonikowsky]

@mconnew , any update on this?

[mconnew]

Still actively working on it.

[Danielku15]

@mconnew Anything the community might be able to help you on this topic?

[mconnew]

At this stage, no. Once I have something working, throwing every contract and service at it and verifying the outcome against what's expected will be the most help. Contributing a test library of contracts, bindings, and expected WSDL's is really useful. Unfortunately this is a big chunk of work which is an all or nothing. Nothing works without every piece, and very little is testable in isolation.

[mconnew]

For anyone watching this issue, I'm about to commit a large amount of code to the feature/wsdl branch to enable contribution of tests. There's a lot of cleanup I need to do in the code, and need to modify the api a little to allow providing a url to expose the metadata page on. But that shouldn't affect the ability to provide tests as I've abstracted all of that away in the 2 initial tests I've created.
The metadata feature had to go into the Primitives package as there was a need for types in Primitives to implement IWsdlExportExtension. That interface then pulled in WsdlExporter and WsdlContractConversionContext which then pulled in pretty much everything. So it didn't make sense to create a separate package.

If you want to contribute test scenarios, it should be relatively simple. Take a look at BasicHttpSimpleServiceTest.cs and NetTcpSimpleServiceTest.cs. The BasicHttpRequestReplyEchoString test body is just this:

        [Fact]
        public async Task BasicHttpRequestReplyEchoString()
        {
            await TestHelper.RunSingleWsdlTestAsync<SimpleEchoService, IEchoService>(new BasicHttpBinding(), _output);
        }

There's a corresponding xml file which contains the single wsdl for the test and is named by convention. They are stored in the Wsdls folder and have the naming convention {TestSourceFileNameWithoutExtension}.{TestName}.xml. There are some test scenarios where there will be multiple tests which all generate the same wsdl, so if the expected file can't be found, it will look for a file called {TestSourceFileNameWithoutExtension}.xml instead.

The test helpers aren't set up to handle multiple endpoints on a single service yet, so please only contribute single endpoint tests for now. There's some work that needs to be done for security mode TransportWithMessageCredentials for HTTP based bindings so if you contribute a test for this, expect it to fail so please add the [Skip] attribute.

The test helpers automatically swap out the endpoint address from the expected wsdl file before comparing. This is to enable multiple variations such as using localhost, or enabling the feature where it copies the hostname from the incoming HTTP Host header and places that in the WSDL endpoint address/location.

Feel free to ask any questions.

[Danielku15]

@mconnew Just gave it a quick shot to use CoreWCF with one of our internal SOAP services. Replaced our ServiceHost with a WebHost using a local build of CoreWCF using feature/wsdl + added System.Web.Services.Description package reference. After some small mistakes from my side at the beginning which I was able to solve looking at the unit tests the service came up as expected. The help website is accessible and the WSDL is returned.

There are only 3 very minor and insignificant differences between the WCF WSDL and the CoreWCF WSDL like:

• CoreWCF does not have a space on shorthand closing tags while WCF had: <xs:element... /> vs <xs:element.../>.
• A xmlns:tns="http://schemas.microsoft.com/2003/10/Serialization/" was generated on one of the xs:schema defining some default types like anyURI, double etc.
• Due to the exception with trailing slashes mentioned below, the soap:address does not have a trailing slash anymore.

Unfortunately I cannot share the test contract but looks already great for our needs.

Findings/Learnings:

1. AddServiceModelMetadata + app.ApplicationServices .GetRequiredService<CoreWCF.Description.ServiceMetadataBehavior>() should be used to configure the ServiceMetadataBehavior. I tried first registering my own behavior through ConfigureServiceHostBase like in classical times.

2. Stumbled over some general ServiceHost migration issues. We have an own dedicated host with a url like "http://*:1234/api/internal/" where the WCF-SOAP service should be listening. Satisfying HttpSys/Kestrel and the CoreWCF bindings is a bit tricky at the start:

   Parameter name: pathMatch
   at Microsoft.AspNetCore.Builder.MapExtensions.Map(IApplicationBuilder app, PathString pathMatch, Action`1 configuration)
   at CoreWCF.Channels.MetadataMiddleware.BuildBranch() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 164
   at CoreWCF.Channels.MetadataMiddleware.EnsureBranchBuilt() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 66
   at CoreWCF.Channels.MetadataMiddleware.BuildBranchAndInvoke(HttpContext request) in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 56
   at CoreWCF.Channels.MetadataMiddleware.<InvokeAsync>d__10.MoveNext() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 51

3. The WSDL url advertised in the website is not advertising the absolute service path for me. I'm using HttpSys with a setup like this:

   Example.cs

var url = "http://localhost:1234/api/internal/"; // coming from somewhere else
var uri = new UriBuilder(url);
uri.Path = uri.Path.TrimEnd('/');

WebHost.CreateDefaultBuilder()
.ConfigureLogging(logging =>
{
    logging.ClearProviders();
    logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
    logging.AddNLog();
})
.ConfigureServices(services =>
{
    services.AddServiceModelServices();
    services.AddServiceModelMetadata();
    services.AddTransient<MyService>();
})
.UseHttpSys(options =>
{
    options.Authentication.Schemes = AuthenticationSchemes.None;
    options.Authentication.AllowAnonymous = true;
    options.AllowSynchronousIO = true;
    options.UrlPrefixes.Add(UrlPrefix.Create(
        "http",
        "*",
        uri.Port,
        ""
    ));
})
.Configure(app =>
{
    app.UseServiceModel(builder =>
    {
        builder.AddService<MyService>();
        builder.AddServiceEndpoint<MyService, IMyService>(
            new CoreWCF.BasicHttpBinding
            {
                // MaxConnections = int.MaxValue, TODO: missing?
                Security =
                {
                    Mode = CoreWCF.Channels.BasicHttpSecurityMode.None
                }
            },
            uri.Uri
        );

        var serviceMetadataBehavior = app.ApplicationServices
            .GetRequiredService<CoreWCF.Description.ServiceMetadataBehavior>();
        serviceMetadataBehavior.HttpGetEnabled = true;
        serviceMetadataBehavior.MetadataExporter.PolicyVersion =
            CoreWCF.Description.PolicyVersion.Default;
    });
})
.Build()

Using the correct URL gives me the right output:

[mconnew]

@Danielku15, I finally had to get around to fixing the base address issue. Because of how ASP.NET Core handles mapping to a sub path, you don't find out what that sub path is until you get the request, and WCF (which CoreWCF came from) has a dependency on knowing that path when building the dispatcher. It's going to take a lot of work and refactoring to change that requirement. So I finally wired up IServiceBuilder.BaseAddresses to help with this. You can just provide the base addresses you are using inside the UseServiceModel delegate by adding them to this collection and that will cause the base address to appear on the help page. Because the service builder is for all services and you might have multiple services each having their own base address, I've also added the overload AddService<TService>(Action<ServiceOptions> options). The ServiceOptions class has a BaseAddresses property which allows you to specify the base addresses just for that service. Try modifying your code to something like this:

    serviceBuilder.AddService<MyService>(options =>
    {
        options.DebugBehavior.HttpHelpPageEnabled = true;
        options.BaseAddresses.Add(new Uri("http://localhost:1234/api/internal"));
    });

You'll also see that I added DebugBehavior to the options class. In WCF, the help page is enabled by default. I'm concerned about adding an extra http page which wasn't there before when people upgrade. I'm going to make the behavior be that you don't get the help page unless you use this new overload, in which case the help page will be on by default. I should probably enable it if you use AddServiceModelMetadata too as you are expecting HTTP pages to be served then. If you want the hostname that was used to request the wsdl to be inserted into it automatically (ie, machine hostname instead of localhost), you can add this behavior to DI to turn that on.

    services.AddSingleton<IServiceBehavior, UseRequestHeadersForMetadataAddressBehavior>();

[Danielku15]

@mconnew Thanks for diving into this topic. I also just quickly looked into the source of this "wrong" URL and I saw that it is computed here: https://github.com/CoreWCF/CoreWCF/blob/25cde9ca34c740ac55f9f6f1ac5372a10fb31a4c/src/CoreWCF.Primitives/src/CoreWCF/Description/ServiceMetadataBehavior.cs#L243

It only asks the host for the URL while it likely should also respect the description.Endpoints[].Address which would hold the URL passed as endpoint for the particular service.

The counter part is here: https://github.com/CoreWCF/CoreWCF/blob/25cde9ca34c740ac55f9f6f1ac5372a10fb31a4c/src/CoreWCF.Primitives/src/CoreWCF/Description/ServiceMetadataExtension.cs#L672-L674

The FindWsdlReference seems to be used a custom WSDL location is set through ServiceMetadataBehavior.ExternalMetadataLocation. which is null in this setup, and GetHttpGetUrl just loads the HttpGetUrl as configured above.

Your proposal seems to try to rely on the current HTTP requset instead of the actual Endpoint address defined for the service. If we would change the HttpGetUrl to point to a mix of the HttpGetUrl derived from the host, and the actual service description we could also solve this issue without any further manual configurations. What are your thoughts about this?

[mconnew]

@Danielku15, that's not a workable option. If you compare it to WCF, you might have a service with a base address http://localhost/services/MyService.svc with 2 endpoints defined. You could have an endpoint defined with the address "BasicHttp" resulting in the url http://localhost/services/MyService.svc/BasicHttp, and a second endpoint defined with the address "WsHttp" resulting in the url http://localhost/services/MyService.svc/WsHttp. The location of the WSDL would be http://localhost/services/MyService.svc?singleWsdl.

Translating that to CoreWCF, you would need to provide the full addresses http://localhost/services/MyService.svc/BasicHttp and http://localhost/services/MyService.svc/WsHttp when adding the service endpoints. Doing the approach you are suggesting would result in the WSDL being at the wrong address. I do have code where a url is mapped in ASP.NET Core where it checks if the address is just the root path, and if so selects the path of the first endpoint instead. This works for some scenarios, but not all. I could add that fallback mechanism to the code which sets HttpGetUrl to the default so that some people will be able to avoid the extra configuration. To help with discovery, I'll then output an info log statement stating that it's using that endpoint address as a fallback due to lack of base addresses being configured..

[allderidge]

I think you know this already @mconnew as your assigned the issue https://github.com/dotnet/runtime/issues/41448 but to help others I've found service contracts that reference an enum type throw a null reference exception for core/net6 runtimes:

System.NullReferenceException: 'Object reference not set to an instance of an object.'

at:

System.Private.DataContractSerialization.dll!System.Runtime.Serialization.SchemaExporter.ExportActualType(System.Xml.XmlQualifiedName typeName, System.Xml.XmlDocument xmlDoc) Line 241

For example the following service contract fails when running the core wcf contract test on it:

[Fact]
public async Task BasicHttpRequestEnumType() => await TestHelper.RunSingleWsdlTestAsync<EnumService, IEnumService>(new BasicHttpBinding(), _output);

[ServiceContract(Namespace = Constants.NS, Name = "EnumService")]
public interface IEnumService
{
    [OperationContract]
    void Accept(TestEnum accept);
}

public enum TestEnum
{
    One,Two,Three
}

[roend83]

I just hit this error trying out the 1.0.0-preview1 package:

An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.InvalidOperationException: An exception was thrown in a call to a WSDL export extension: CoreWCF.Description.DataContractSerializerOperationBehavior\r\n contract: http://tempuri.org/:IApslService ----> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ----> System.InvalidOperationException: DataContract for type 'System.Int32[]' cannot be added to DataContractSet since type 'System.Collections.Generic.List1[[System.Int32, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]' with the same data contract name 'ArrayOfint' in namespace 'http://schemas.microsoft.com/2003/10/Serialization/Arrays' is already present and the contracts are not equivalent. at System.Runtime.Serialization.DataContractSet.InternalAdd(XmlQualifiedName name, DataContract dataContract) at System.Runtime.Serialization.DataContractSet.AddClassDataContract(ClassDataContract classDataContract) at System.Runtime.Serialization.DataContractSet.InternalAdd(XmlQualifiedName name, DataContract dataContract) at System.Runtime.Serialization.DataContractSet.Add(DataContract dataContract) --- End of inner ExceptionDetail stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Span1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) at CoreWCF.Runtime.Serialization.DataContractSetEx.Add(Type type) at CoreWCF.Runtime.Serialization.XsdDataContractExporterEx.AddType(Type type) at CoreWCF.Runtime.Serialization.XsdDataContractExporterEx.Export(Type type) at CoreWCF.Description.MessageContractExporter.ExportType(Type type, String partName, String operationName, XmlSchemaType& xsdType) at CoreWCF.Description.DataContractSerializerMessageContractExporter.ExportBody(Int32 messageIndex, Object state) at CoreWCF.Description.MessageContractExporter.ExportMessage(Int32 messageIndex, Object state) at CoreWCF.Description.MessageContractExporter.ExportMessageContract() at CoreWCF.Description.DataContractSerializerOperationBehavior.CoreWCF.Description.IWsdlExportExtension.ExportContract(WsdlExporter exporter, WsdlContractConversionContext contractContext) at CoreWCF.Description.WsdlExporter.CallExtension(WsdlContractConversionContext contractContext, IWsdlExportExtension extension) --- End of inner ExceptionDetail stack trace --- at CoreWCF.Description.WsdlExporter.CallExtension(WsdlContractConversionContext contractContext, IWsdlExportExtension extension) at CoreWCF.Description.WsdlExporter.CallExportContract(WsdlContractConversionContext contractContext) at CoreWCF.Description.WsdlExporter.ExportContract(ContractDescription contract) at CoreWCF.Description.WsdlExporter.ExportEndpoint(ServiceEndpoint endpoint, XmlQualifiedName wsdlServiceQName, BindingParameterCollection bindingParameters) at CoreWCF.Description.WsdlExporter.ExportEndpoints(IEnumerable1 endpoints, XmlQualifiedName wsdlServiceQName, BindingParameterCollection bindingParameters) at CoreWCF.Description.ServiceMetadataBehavior.MetadataExtensionInitializer.GenerateMetadata() at CoreWCF.Description.ServiceMetadataExtension.EnsureInitialized() at CoreWCF.Description.ServiceMetadataExtension.HttpGetImpl.InitializationData.InitializeFrom(ServiceMetadataExtension extension) at CoreWCF.Description.ServiceMetadataExtension.HttpGetImpl.GetInitData() at CoreWCF.Description.ServiceMetadataExtension.HttpGetImpl.TryHandleMetadataRequestAsync(HttpContext requestContext, IQueryCollection queries) at CoreWCF.Description.ServiceMetadataExtension.HttpGetImpl.ProcessHttpRequest(HttpContext requestContext) at CoreWCF.Description.ServiceMetadataExtension.HttpGetImpl.HandleRequest(HttpContext httpContext)`

[cpx86]

@roend83 I hit the same issue as described in this PR discussion. @mconnew explained the underlying problem further very well there - it's apparently not trivial to solve. https://github.com/CoreWCF/CoreWCF/pull/550#issuecomment-1083106378

If you can modify the underlying data contracts though, there is a workaround that solved it for me - basically just make sure that you don't have multiple types within the same service contract which would resolve to the same WSDL type. So for example, don't mix int[], List<int>, IEnumerable<int>, etc within the same contract, but settle for one of them and use it everywhere. For clients of the service it shouldn't make any difference, since the actual WSDL contract will be identical regardless of which type you choose (it will always resolve to ArrayOfint).

[roend83]

@cpx86 Changing this to a List does get rid of the error.

[SayedMohammadHaider]

@mconnew @cpx86 Can you please help me on this. Is there any other way to fix this issue without modifying data contracts? as we have existing data contracts and we cannot modify that.

[rashmidusane]

I am trying to add a service reference for CoreWCF service in WCF (.Net framework) client but it is not able to discover it. Is it because the Mex endpoint is not supported? If I start the service in another Visual studio instance and then try to add a service reference with basichttp, I get the following error. Any workaround to get it discovered?

Thanks!

[mconnew]

@rashmidusane, if you check the output from starting the service, it should tell you the path that the WSDL is listening on. You can open that path in a browser and should see the help page with a link to the wsdl. Use the url which includes ?wsdl or ?singleWsdl in the Add Service Reference dialog. The key thing is to verify that you can get to the wsdl help page to know you have the correct path.

[rashmidusane]

Thank you very much @mconnew! that works!!

[SKO85]

@mconnew Just gave it a quick shot to use CoreWCF with one of our internal SOAP services. Replaced our ServiceHost with a WebHost using a local build of CoreWCF using feature/wsdl + added System.Web.Services.Description package reference. After some small mistakes from my side at the beginning which I was able to solve looking at the unit tests the service came up as expected. The help website is accessible and the WSDL is returned.

There are only 3 very minor and insignificant differences between the WCF WSDL and the CoreWCF WSDL like:

• CoreWCF does not have a space on shorthand closing tags while WCF had: <xs:element... /> vs <xs:element.../>.
• A xmlns:tns="http://schemas.microsoft.com/2003/10/Serialization/" was generated on one of the xs:schema defining some default types like anyURI, double etc.
• Due to the exception with trailing slashes mentioned below, the soap:address does not have a trailing slash anymore.

Unfortunately I cannot share the test contract but looks already great for our needs.

Findings/Learnings:

1. AddServiceModelMetadata + app.ApplicationServices .GetRequiredService<CoreWCF.Description.ServiceMetadataBehavior>() should be used to configure the ServiceMetadataBehavior. I tried first registering my own behavior through ConfigureServiceHostBase like in classical times.
2. Stumbled over some general ServiceHost migration issues. We have an own dedicated host with a url like "http://*:1234/api/internal/" where the WCF-SOAP service should be listening. Satisfying HttpSys/Kestrel and the CoreWCF bindings is a bit tricky at the start:

Parameter name: pathMatch
   at Microsoft.AspNetCore.Builder.MapExtensions.Map(IApplicationBuilder app, PathString pathMatch, Action`1 configuration)
   at CoreWCF.Channels.MetadataMiddleware.BuildBranch() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 164
   at CoreWCF.Channels.MetadataMiddleware.EnsureBranchBuilt() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 66
   at CoreWCF.Channels.MetadataMiddleware.BuildBranchAndInvoke(HttpContext request) in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 56
   at CoreWCF.Channels.MetadataMiddleware.<InvokeAsync>d__10.MoveNext() in D:\Dev\Git\CoreWCF\src\CoreWCF.Primitives\src\CoreWCF\Channels\MetadataMiddleware.cs:line 51

3. The WSDL url advertised in the website is not advertising the absolute service path for me. I'm using HttpSys with a setup like this:

Example.cs

Using the correct URL gives me the right output:

I have the exact same issue where I have multiple hostnames with relative paths referencing to the same service. I host the service in a container and there are 2 ingress endpoints pointing to it. When I go to the main service URL I see the help page what points to the root of the hostname url for the WSDL which is not correct.

Example path: http://localhost:8080/services/MyService?wsdl but it shows http://localhost:8080/MyService?wsdl so the /services/ path is not included.

Is there a fix or workaround for this?