Low compression issue - IIS & ASP.Net Core

[Tratcher]

From @sylvaincaillot on February 9, 2017 3:30

Hello, I have developed the same application which delivers a JSON file in 2 different projects. One is based on ASP.Net MVC4 and the other is based on ASP.Net Core 1.1.0. The JSON generated by my controller is around 5Mo and its content-type is text/plain when returned to the client.

However when publishing the 2 apps, i have a difference of sizes for this file if it is compressed or not on both plateforms.

For ASP.Net Core, I have even tried to enable the compression middleware in the Startup.cs to see the difference:

    services.AddResponseCompression(
        options =>
            options.MimeTypes = ResponseCompressionMimeTypes.Defaults);

    app.UseResponseCompression()
       .UseStaticFiles();

This is the summary of my tests for my original 5Mo JSON returned to the client:

On my local IIS 7.5 with dynamic and static compression enabled, size of my JSON on:

• MVC4: 256Ko
• ASP.Net Core default: 1.3Mo
• ASP.Net core with compression middleware: 869Ko

On my production IIS 7.5 with dynamic and static compression enabled, size of my JSON on:

• MVC4: 136Ko
• ASP.Net Core default: 675Ko
• ASP.Net core with compression middleware: 869Ko

For MVC4, it seems that it is using full IIS compression capabilities.

For ASP.Net Core, when the compression is enabled in the Startup.cs, the size of the file is constant but not as small as it could be.

For ASP.Net Core, when the compression is not enabled in Startup.cs, the file is gzip by default but I am not sure it is even using the compression capabilities of the IIS server.

My question is what could I do to obtain a compressed JSON in ASP.Net Core as small as the one I obtained on my older plateform based on MVC4.

I tried to use RunIISPipeline as suggested here but it is no more available.

What are the best practices to publish on IIS an ASP.Net Core app and make sure that the compression is optimal?

Thanks

Sylvain

Copied from original issue: aspnet/Hosting#934

[Tratcher]

Start here: https://docs.microsoft.com/en-us/aspnet/core/performance/response-compression

services.Configure<GzipCompressionProviderOptions>(options => 
{
    options.Level = CompressionLevel.Optimal;
});

I'm surprised the MVC4 ration was that much higher. It may be because MVC4 buffered the full response before compression and could optimize better, where ASP.NET Core streams the response.

@pan-wang @shirhatti any suggestions for IIS dynamic compression?

[sylvaincaillot]

With the Optimal compression, it's a bit better with a 627KB response file even if the TTFB is a bit longer.

However, we are still far from the MVC4 score. Just to fully understand: by adding the compression middleware in my Core app, are we using the IIS compression capabilities or the compression only occurs on the Core side?

Is there any mixed mode as IIS compression seems quite robust?

Thank you for your help

Sylvain

[Tratcher]

The Core compression middleware is independent of IIS and causes the IIS compression to be bypassed.

[sylvaincaillot]

OK understood, so If i remove the Core compression middleware, it should use the IIS compression instead, right? So why do I have such difference between my MVC4 app and my ASP.Net Core default app (136Ko vs 675Ko)? Would it be related to the buffer of the full reponse in MVC4 as you suggested?

Thanks S.

[Tratcher]

Yes, removing the middleware causes you to use the IIS compression instead. Not sure why it's different from MVC4, I'll defer to the IIS folks ( @pan-wang @shirhatti ) for that analysis.

Buffering in Core likely wouldn't help as the response would still be streamed across to IIS. MVC4 had an advantage of running inside of IIS rather than behind it and could hand it the entire buffer direclty.

[sylvaincaillot]

Just to give some further details concerning the response:

This is what I have for MVC4 app on my production server and the 136KB compressed JSOn file:

Age:0
Cache-Control:private
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Thu, 09 Feb 2017 05:06:32 GMT
Server:cloudflare-nginx
Transfer-Encoding:chunked
Vary:Accept-Encoding
Via:1.1 varnish
Via:1.1 varnish
X-AspNet-Version:4.0.30319
X-AspNetMvc-Version:4.0
X-Cache:MISS
X-Cache-Hits:0
X-Powered-By:ASP.NET

The same MVC4 app hosted on my local IIS with a 256KB compressed JSON file:

Cache-Control:private
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Thu, 09 Feb 2017 06:10:48 GMT
Server:Microsoft-IIS/7.5
Transfer-Encoding:chunked
Vary:Accept-Encoding
X-AspNet-Version:4.0.30319
X-AspNetMvc-Version:4.0
X-Powered-By:ASP.NET 

This is what i have for my ASP.Net Core app on my production server with a 627KB compressed JSON file:

Accept-Ranges:bytes
Age:0
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/plain; charset=utf-8
Date:Thu, 09 Feb 2017 05:13:16 GMT
Server:cloudflare-nginx
Transfer-Encoding:chunked
Via:1.1 varnish
X-Cache:MISS
X-Cache-Hits:0

and the same Core app hosted on my local IIS with a 627KB compressed JSON file:

Content-Encoding:gzip
Content-Type:text/plain; charset=utf-8
Date:Thu, 09 Feb 2017 06:09:26 GMT
Server:Kestrel
Transfer-Encoding:chunked
X-Powered-By:ASP.NET

There are some slight differences between the 2. The content-type of the responses is different, Accept-Encoding is only present for the MVC4. Could it be an issue? As we are using NGinx for the production server, could it also prevent the middleware from compressing the response in the case of the Core app as suggested in the link above or here?

[shirhatti]

@jhkimnew Is this consistent with your findings? I know you recently looked into this.

[jhkimnew]

IIS has two compression modes, one for Dynamic compression and the other for Static compression. Recently I noticed that the two compression mode shows different compression result because Dynamic compression has some different logic to handle data in buffer.

@sylvaincaillot
Can you check if there is any difference between your production server and your local IIS and test it again in the same condition? If you can't find any difference, you just try again with enabling Static compression only and disabling Dynamic compressions for your local IIS server. If the test result is the same, you can try again with only enabling Dynamic compression and disabling Static compression for your local IIS.

[sylvaincaillot]

Thank you @jhkimnew for your reply. On my side I have tried to reduce as much as I can the size of my JSON, i am now dealing with a 2.2MB file. There are no difference between my locall IIS 7.5 and my production server, both dynamic and static compression are enabled.

I tried as suggested: If I add the compression middleware (Fastest), I haev now a 551KB file. With the optimal compression, I end up with a 415KB but it takes longer to compress and toatl response time at the end is not better.

If I remove the compression middleware and rely solely on IIS compression i have:

• Dynamic compression only: 739KB
• Static compression only: 2.2MB (No compression)

On my production server, I have the same results while using the compression middleware and reach a 439KB JSON file when using the Dynamic Compression.

So it is getting better but far far away from MVC4 scores. I start regretting my old app and its performances...

Sylvain

[muratg]

We are closing this issue because no further action is planned for this issue. If you still have any issues or questions, please log a new issue with any additional details that you have.