SyndicationItem.Load + FileBufferingReadStream (via Request.EnableBuffering()) fails with "Root element is missing" exception

[bergner]

At https://github.com/bergner/netcore-bugs/ I've provided a webapi example that reproduces this problem. It uses netcoreapp2.1.

$ git clone https://github.com/bergner/netcore-bugs.git
$ cd netcore-bugs/RequestBuffering
$ dotnet build
$ dotnet run RequestBuffering

This starts a service with POST and PUT support on /api/Values/. The POST endpoint uses [FromBody] SyndicationItem and the PUT endpoint uses [FromBody] XmlElement. In Startup.cs context.Request.EnableBuffering() is called (before app.UseMvc()). The request buffering is a key component to the issue here.

app.Use(next => context => {
    Console.WriteLine("BODY IS OF TYPE: {0}", context.Request.Body.GetType());
    context.Request.EnableBuffering();
    //(new StreamReader(context.Request.Body)).ReadToEnd();
    //context.Request.Body.Position = 0;
    Console.WriteLine("BODY IS OF TYPE: {0}", context.Request.Body.GetType());
    return next(context);
});

This causes ASP.NET Core to switch the context.Request.Body from a Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream object to a Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream object.

The service has a AtomEntryInputFormatter class which is used to parse the incoming HTTP request into a SyndicationItem object (using XmlReader, see below). It is also using an XmlSerializerInputFormatter to support the PUT requests (which are used for comparison here).

using System;
using System.IO;
using System.Reflection;
using System.ServiceModel.Syndication;
using System.Text;
using System.Threading.Tasks;
using System.Xml;
using System.Xml.Serialization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Formatters;

namespace RequestBuffering
{
    public class AtomEntryInputFormatter : XmlSerializerInputFormatter
    {
        public AtomEntryInputFormatter(MvcOptions options) : base(options)
        {
            SupportedMediaTypes.Clear();
            SupportedMediaTypes.Add("application/atom+xml;type=entry");
        }

        protected override bool CanReadType(Type dataType)
        {
            Console.WriteLine("CAN READ TYPE? {0} --> {1}", dataType, typeof(SyndicationItem).IsAssignableFrom(dataType));
            return typeof(SyndicationItem).IsAssignableFrom(dataType);
        }

        public override async Task<InputFormatterResult> ReadRequestBodyAsync(InputFormatterContext context,
            Encoding encoding)
        {
            try {
                Console.WriteLine("PARSING STREAM {0}", context.HttpContext.Request.Body.GetType());
                using (XmlReader reader = XmlReader.Create(context.HttpContext.Request.Body))
                {
                    SyndicationItem item = SyndicationItem.Load(reader);
                    Console.WriteLine("SUCCESSFULLY PARSED: {0}", item.Title.Text);
                    return await InputFormatterResult.SuccessAsync(item);
                }
            } catch (Exception e) {
                Console.WriteLine("CAUGHT EXCEPTION WHILE PARSING: {0}", e.Message);
                throw e;
            }
        }
    }
}

A sample input xml below:

$ cat /tmp/entry.xml
<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="http://www.w3.org/2005/Atom"><title>test</title></entry>

Using PUT to the XmlElement endpoint works ok. I get a 200 response and the controller gets the expected data (which can be seen in Console.WriteLine output from the server).

$ curl -X PUT -k -H "Content-Type: application/atom+xml" --data-binary @/tmp/entry.xml https://localhost:5001/api/Values/

The POST request fails with a 400 Bad request:

$ curl -X POST -k -H "Content-Type: application/atom+xml" --data-binary @/tmp/entry.xml https://localhost:5001/api/Values/

And the log shows an exception:

info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 POST https://localhost:5001/api/Values/ application/atom+xml;type=entry 110
BODY IS OF TYPE: Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream
BODY IS OF TYPE: Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Route matched with {action = "Post", controller = "Values"}. Executing action RequestBuffering.Controllers.ValuesController.Post (RequestBuffering)
CAN READ TYPE? System.ServiceModel.Syndication.SyndicationItem --> True
PARSING STREAM Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream
CAUGHT EXCEPTION WHILE PARSING: Root element is missing.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
      Executing ObjectResult, writing value of type 'Microsoft.AspNetCore.Mvc.SerializableError'.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
      Executed action RequestBuffering.Controllers.ValuesController.Post (RequestBuffering) in 48.2919ms
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
System.InvalidOperationException: There was an error generating the XML document. ---> System.ArgumentException: The empty string '' is not a valid local name.
   at System.Xml.XmlWellFormedWriter.WriteStartElement(String prefix, String localName, String ns)
   at System.Xml.XmlWriter.WriteStartElement(String localName)
   at Microsoft.AspNetCore.Mvc.Formatters.Xml.SerializableErrorWrapper.WriteXml(XmlWriter writer)
   at System.Xml.Serialization.XmlSerializationWriter.WriteSerializable(IXmlSerializable serializable, String name, String ns, Boolean isNullable, Boolean wrapped)
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)
   at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o)
   at Microsoft.AspNetCore.Mvc.Formatters.XmlSerializerOutputFormatter.Serialize(XmlSerializer xmlSerializer, XmlWriter xmlWriter, Object value)
   at Microsoft.AspNetCore.Mvc.Formatters.XmlSerializerOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultAsync(IActionResult result)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeResultFilters()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Workaround

I have found 2 workarounds for this problem:

1. Don't call XmlReader reader = XmlReader.Create(context.HttpContext.Request.Body) directly in preparation for SyndicationItem.Load(reader). Instead wrap the body stream with new StreamReader(context.HttpContext.Request.Body) then use that with XmlReader.Create.
2. Make sure that the request body has been read at least once BEFORE you get to the XmlReader / input formatter. Commenting out the two lines in Startup.cs above makes the problem go away.

Removing EnableBuffering() also makes the problem go away but there are many circumstances where you want/need to have request buffering enabled so it is not a viable workaround.

Expected result

The reading / parsing from context.HttpContext.Request.Body should have the same behavior regardless if EnableBuffering is used or not, and when buffering is used it should not matter it the body has been read prior to reaching SyndicationItem.Load.

[StephenBonikowsky]

@imcarolwang We need to compare the raw soap body content of the PUT and POST calls and see what the difference is. Please use the example provided to create a repro in order to get this info.

[anhadi2]

We are building a CoreWCF service and faced this issue when we enabled request buffering.

Steps:

1. In middleware, call HttpContext.Request.EnableBuffering();
2. Get request stream via HttpContext.Request.InputStream, it returns stream of type FileBufferingReadStream
3. Create a XmlReader via XmlReader xmlReader = XmlReader.Create(stream).
4. Invoking XmlReader.Read() throws exception "Root element is missing"