Add a Priority to each Response Compression Provider

[RehanSaeed]

Tomasz Pęczek blog post shows that if you add GZIP and Brotli response compression providers, GZIP is always the one used even if the browser supports Brotli

The middleware takes the advertised compressions, sorts them by quality if present and chooses the first one for which provider exists. As browser generally don't provide any quality values (in other words they will be equally happy to accept any of the supported ones) the gzip provider always wins because it is always first on the advertised list.

If a quality value is not provided, we need a method of assigning a priority ourselves.

[jbayardo]

We need this for RavenDB. If we were to make a PR for the feature, what would the ETA be for delivery?

[Tratcher]

Any improvements here would go into our next 2.1.0 preview later this year.

This functionality could be achieved today by overriding ResponseCompressionProvider.GetCompressionProvider.

[speige]

+1

[muratg]

@davidfowl thoughts?

[speige]

i coded up a possible solution, but i haven't tested it yet. i'm happy to submit a pull request if you'd like.

i believe this will default to the order the providers were registered but allow quality request headers to override. Consumers of the middleware will need to register both brotli & gzip, because gzip is only added automatically if providers is empty.

    public class CustomResponseCompressionProvider : ResponseCompressionProvider
    {
        protected readonly ICompressionProvider[] _providers;

        public CustomResponseCompressionProvider(IServiceProvider services, IOptions<ResponseCompressionOptions> options) : base(services, options)
        {
            _providers = (ICompressionProvider[])typeof(ResponseCompressionProvider).GetField("_providers", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(this);
        }
        public override ICompressionProvider GetCompressionProvider(HttpContext context)
        {
            IList<StringWithQualityHeaderValue> unsorted;

            // e.g. Accept-Encoding: gzip, deflate, sdch
            var accept = context.Request.Headers[HeaderNames.AcceptEncoding];
            if (!StringValues.IsNullOrEmpty(accept)
                && StringWithQualityHeaderValue.TryParseList(accept, out unsorted)
                && unsorted != null && unsorted.Count > 0)
            {
                // TODO PERF: clients don't usually include quality values so this sort will not have any effect. Fast-path?
                var sorted = unsorted
                    .GroupBy(s => s.Quality.GetValueOrDefault(1) > 0)
                    .OrderByDescending(s => s.Key).ToList();

                foreach (var qualityGroup in sorted)
                {
                    // Uncommon but valid option
                    if (qualityGroup.Any(x => StringSegment.Equals("*", x.Value, StringComparison.Ordinal)))
                    {
                        // Any
                        return _providers[0];
                    }

                    // There will rarely be more than three providers, and there's only one by default
                    foreach (var provider in _providers)
                    {
                        if (qualityGroup.Any(x => StringSegment.Equals(provider.EncodingName, x.Value, StringComparison.Ordinal)))
                        {
                            return provider;
                        }
                    }

                    // Uncommon but valid option
                    if (qualityGroup.Any(x => StringSegment.Equals("identity", x.Value, StringComparison.Ordinal)))
                    {
                        // Any
                        return _providers[0];
                    }
                }
            }

            return null;
        }
    }

It can be used in the ConfigureServices method of Startup.cs like this:

            services.Configure<GzipCompressionProviderOptions>(options => options.Level = System.IO.Compression.CompressionLevel.Fastest);
            services.Configure((ResponseCompressionOptions options) =>
            {
                options.Providers.Add<BrotliCompressionProvider>();
                options.Providers.Add<GzipCompressionProvider>();
                options.EnableForHttps = true;
                options.MimeTypes = new[]
                {
                    // Default
                    "text/plain",
                    "text/css",
                    "application/javascript",
                    "text/html",
                    "application/xml",
                    "text/xml",
                    "application/json",
                    "text/json",

                    // Custom
                    "application/atom+xml",
                    "application/xaml+xml",
                    "application/svg+xml",
                    "image/svg+xml",
                    "application/vnd.ms-fontobject",
                    "font/otf"
                };
            });
            services.TryAddSingleton<IResponseCompressionProvider, CustomResponseCompressionProvider>();

The normal UseResponseCompression(); in Startup.cs Configure should still work.

[Tratcher]

The bit about Accept-Encoding: identity doesn't look right, but the rest seems workable.

[rsantosdev]

I updated my project to 2.1 today. Still with issue when using both GZip and Custom (Brotli) in my case.

The problem: 1 - if gzip compression is present, it always selected because chrome does not send priority, so gzip always wins. 2 - Testing a little more br (for Brotli) is only sent on https, not on http, so we need to support both scenarios.

The Solution: I took the code of @speige as base and did some modifications. Main modification was to keep order where prodivers are specified on Startup.cs and calling base class in case. Also the services.TryAddSingleTon needs to happen before services.AddResponseCompression for those using the helper.

The CustomProvider:

public class CustomResponseCompressionProvider : ResponseCompressionProvider {
        protected readonly ICompressionProvider[] _providers;

        public CustomResponseCompressionProvider(IServiceProvider services, IOptions<ResponseCompressionOptions> options) : base(services, options) {
            // On base class _providers is private
            _providers = (ICompressionProvider[])typeof(ResponseCompressionProvider)
                .GetField("_providers", BindingFlags.NonPublic | BindingFlags.Instance)
                .GetValue(this);
        }

        public override ICompressionProvider GetCompressionProvider(HttpContext context) {

            // e.g. Accept-Encoding: gzip, deflate, sdch
            var accept = context.Request.Headers[HeaderNames.AcceptEncoding];
            if (!StringValues.IsNullOrEmpty(accept)
                && StringWithQualityHeaderValue.TryParseList(accept, out var unsorted)
                && unsorted != null && unsorted.Count > 0) {

                foreach (var provider in _providers) {
                    if (unsorted.Any(x => StringSegment.Equals(provider.EncodingName, x.Value, StringComparison.Ordinal))) {
                        return provider;
                    }
                }
            }

            return base.GetCompressionProvider(context);
        }
    }

Registering on Startup.cs

services.TryAddSingleton<IResponseCompressionProvider, CustomResponseCompressionProvider>();
            services.AddResponseCompression(opts => {
                opts.Providers.Add<BrotliCompressionProvider>();
                opts.Providers.Add<GzipCompressionProvider>();
                opts.EnableForHttps = true;
            });

Since, I'm already here, code for BrotliCompressionProvider:

public class BrotliCompressionProvider : ICompressionProvider {
        public string EncodingName => "br";
        public bool SupportsFlush => true;

        public Stream CreateStream(Stream outputStream) => new BrotliStream(
            outputStream,
            CompressionMode.Compress,
            leaveOpen: false);
    }