Allow exclude MIME types for HTTP compression

andrewslavin commented 6 years ago

Add ability to configure response compression middleware to compress responses with all MIME types EXCEPT the specified list. Achieved through adding a new property to ResponseCompressionOptions that allows to control how the specified list of MIME types will be used:

Compress responses with MIME type in the list (current behaviour), or
Compress responses with MIME type NOT in the list (new behaviour)

dnfclas commented 6 years ago

All CLA requirements met.

Tratcher commented 6 years ago

@glennc This is a curious design choice. I would expect building an exhaustive exclusion list would be just as hard as building an exhaustive inclusion list. Would supporting wildcards make the existing inclusion list design more tenable? https://github.com/aspnet/BasicMiddleware/issues/121

andrewslavin commented 6 years ago

Hi @Tratcher.

I would expect building an exhaustive exclusion list would be just as hard as building an exhaustive inclusion list.

My company runs an online service, which among other things stores and serves files. We compress them for storage, and in the database we record both original and compressed file size. We also record the MIME type. A few years ago I analysed the database (a few million records) to find which MIME types compress well and which don't. Based on that, I configured IIS compression as follows:

<dynamicTypes>
    <add mimeType="application/octet-stream" enabled="false" />
    <add mimeType="application/pdf" enabled="false" />
    <add mimeType="application/vnd.ms-excel.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.addin.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.sheet.binary.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.sheet.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.addin.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.presentation.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.slide.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.slideshow.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.document.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.document.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.template.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.presentation" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.slide" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.slideshow" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.template" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.template" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.wordprocessingml.document" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.wordprocessingml.template" enabled="false" />
    <add mimeType="application/vnd.ms-xpsdocument" enabled="false" />
    <add mimeType="application/x-captivate" enabled="false" />
    <add mimeType="application/x-compressed" enabled="false" />
    <add mimeType="application/x-dwf" enabled="false" />
    <add mimeType="application/x-gzip" enabled="false" />
    <add mimeType="application/x-rar" enabled="false" />
    <add mimeType="application/x-rar-compressed" enabled="false" />
    <add mimeType="application/x-zip" enabled="false" />
    <add mimeType="application/x-zip-compressed" enabled="false" />
    <add mimeType="application/zip" enabled="false" />
    <add mimeType="audio/mp3" enabled="false" />
    <add mimeType="audio/mp4" enabled="false" />
    <add mimeType="audio/x-ms-wma" enabled="false" />
    <add mimeType="image/gif" enabled="false" />
    <add mimeType="image/jpeg" enabled="false" />
    <add mimeType="image/pjpeg" enabled="false" />
    <add mimeType="image/vnd.dgn" enabled="false" />
    <add mimeType="image/vnd.ms-modi" enabled="false" />
    <add mimeType="image/x-png" enabled="false" />
    <add mimeType="video/mpeg" enabled="false" />
    <add mimeType="video/quicktime" enabled="false" />
    <add mimeType="video/x-ms-wmv" enabled="false" />
    <add mimeType="*/*" enabled="true" />
</dynamicTypes>
<staticTypes>
    <add mimeType="application/octet-stream" enabled="false" />
    <add mimeType="application/pdf" enabled="false" />
    <add mimeType="application/vnd.ms-excel.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.addin.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.sheet.binary.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.sheet.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-excel.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.addin.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.presentation.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.slide.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.slideshow.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-powerpoint.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.document.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.document.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.template.12" enabled="false" />
    <add mimeType="application/vnd.ms-word.template.macroEnabled.12" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.presentation" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.slide" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.slideshow" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.presentationml.template" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.spreadsheetml.template" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.wordprocessingml.document" enabled="false" />
    <add mimeType="application/vnd.openxmlformats-officedocument.wordprocessingml.template" enabled="false" />
    <add mimeType="application/vnd.ms-xpsdocument" enabled="false" />
    <add mimeType="application/x-captivate" enabled="false" />
    <add mimeType="application/x-compressed" enabled="false" />
    <add mimeType="application/x-dwf" enabled="false" />
    <add mimeType="application/x-gzip" enabled="false" />
    <add mimeType="application/x-rar" enabled="false" />
    <add mimeType="application/x-rar-compressed" enabled="false" />
    <add mimeType="application/x-zip" enabled="false" />
    <add mimeType="application/x-zip-compressed" enabled="false" />
    <add mimeType="application/zip" enabled="false" />
    <add mimeType="audio/mp3" enabled="false" />
    <add mimeType="audio/mp4" enabled="false" />
    <add mimeType="audio/x-ms-wma" enabled="false" />
    <add mimeType="image/gif" enabled="false" />
    <add mimeType="image/jpeg" enabled="false" />
    <add mimeType="image/pjpeg" enabled="false" />
    <add mimeType="image/vnd.dgn" enabled="false" />
    <add mimeType="image/vnd.ms-modi" enabled="false" />
    <add mimeType="image/x-png" enabled="false" />
    <add mimeType="video/mpeg" enabled="false" />
    <add mimeType="video/quicktime" enabled="false" />
    <add mimeType="video/x-ms-wmv" enabled="false" />
    <add mimeType="*/*" enabled="true" />
</staticTypes>

As you can see, I excluded everything that does not compress well, and then said "compress everything else". The rationale was:

For serving files, this list properly decides what makes sense to compress.
For serving usual web content (html, javascript, css) - just compress everything, most likely it will be text anyway.

andrewslavin commented 6 years ago

Closing this for now, will try to implement MimeTypeFilter as described in #121.

aspnet / BasicMiddleware

Allow exclude MIME types for HTTP compression #311