SimonbJohnson
commented
4 years ago This is great research. Thank you. I will update the server this Friday and will enable this!
Open fititnt opened 4 years ago
SimonbJohnson
commented
4 years ago This is great research. Thank you. I will update the server this Friday and will enable this!
SimonbJohnson
commented
4 years ago I tried implementing this today, but ran into trouble that the module is not recognised with apache. i tried upgrading apache on a test server and still then it is not recognised.
a2enmod brotli ERROR: Module brotli does not exist!
I tried implementing the AddOutputFilterByType, but that was not recognised either, so I will investigate more.
Test server is hosted here: http://34.229.180.70/
fititnt
commented
4 years ago Can you return the output of the commands? With the exact OS version I may try a quick install on some VPS just to check this.
# OS Version
cat /etc/*release
# List from where the apache is installed
apt policy apache2
# List available modules, since apache2ctl -M only return enabled
ls /etc/apache2/mods-available/@SimonbJohnson @Rydela This comment is a reference for https://github.com/SimonbJohnson/quickX3/issues/66#issuecomment-650211579 and this https://github.com/SimonbJohnson/quickX3/issues/66#issuecomment-650294799. I will post here on the #85 since is somewhat a way to debug who implement.
maybe this is a result of testing the `Enable brotli compression on hxldash.com
.jsonContent-Type: application/jsonRefs: https://stackoverflow.com/a/18984239/894546
One way to test the content is with compression is using curl --compressed --head https://example.com (-H or --head show just the HTTP headers). Here the example at this very moment:
TL:DR: /static/geoms/topojson/AAA/N/geom.json is not returning Content-Encoding: gzip, but other files are.
# fititnt at fititnt-bravo in ~ [20:22:27]
$ curl --compressed --head https://hxldash.com/static/geoms/topojson/BRA/2/geom.json
HTTP/1.1 200 OK
Date: Fri, 26 Jun 2020 23:23:06 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Fri, 26 Jun 2020 10:36:32 GMT
ETag: "2d498e-5a8fa48559a72"
Accept-Ranges: bytes
Content-Length: 2967950
Content-Type: application/json
# fititnt at fititnt-bravo in ~ [20:23:07]
$ curl --compressed --head https://hxldash.com/static/libs/hxlbites/hxl.js
HTTP/1.1 200 OK
Date: Fri, 26 Jun 2020 23:23:26 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Wed, 20 Feb 2019 12:02:09 GMT
ETag: "cb4a-582522099e320-gzip"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 11484
Content-Type: application/javascript
# fititnt at fititnt-bravo in ~ [20:23:30]
$ curl --compressed --head https://hxldash.com/view/341
HTTP/1.1 200 OK
Date: Fri, 26 Jun 2020 23:23:44 GMT
Server: Apache/2.4.29 (Ubuntu)
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
# fititnt at fititnt-bravo in ~ [20:23:46]
$ curl --version
curl 7.69.1 (x86_64-conda_cos6-linux-gnu) libcurl/7.69.1 OpenSSL/1.1.1g zlib/1.2.11 libssh2/1.9.0
Release-Date: 2020-03-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTPS-proxy IPv6 Kerberos Largefile libz NTLM NTLM_WB SPNEGO SSL TLS-SRP UnixSockets
From https://daniel.haxx.se/blog/tag/brotli/ (NOVEMBER 29, 2017, curl v7.57.0) says:
Brotli compression The popular browsers have supported brotli compression method for a while and it has already become widely supported by servers.
Now, curl supports it too and the command line tool’s –compressed option will ask for brotli as well as gzip, if your build supports it. Similarly, libcurl supports it with its CURLOPT_ACCEPT_ENCODING option. The server can then opt to respond using either compression format, depending on what it knows.
According to CertSimple, who ran tests on the top-1000 sites of the Internet, brotli gets contents 14-21% smaller than gzip.
Since depending of the version of the curl as soon as brotli is enabled, the result may be the most efficient, the more specific question from https://stackoverflow.com/a/18984239/894546 instead of use --compressed may be need to debug:
## GZip only
curl -sH 'Accept-encoding: gzip' --head https://hxldash.com/static/geoms/topojson/BRA/2/geom.json
curl -sH 'Accept-encoding: gzip' --head https://hxldash.com/static/libs/hxlbites/hxl.js
curl -sH 'Accept-encoding: gzip' --head https://hxldash.com/view/341
## Request similar to a browser like Chrome
curl -sH 'Accept-encoding: gzip, deflate, br' --head https://hxldash.com/static/geoms/topojson/BRA/2/geom.json
curl -sH 'Accept-encoding: gzip, deflate, br' --head https://hxldash.com/static/libs/hxlbites/hxl.js
curl -sH 'Accept-encoding: gzip, deflate, br' --head https://hxldash.com/view/341 Here explain one way https://webmasters.stackexchange.com/a/105061/42038. Requires "Large request rows" option enabled.
This may be good to comprare compressed vs umcompressed. Here a quick print screen, sorted by request size
refs
Since the response headers of hxldash.com give a hint that the server version is
Server: Apache/2.4.29 (Ubuntu)and Apache Module mod_brotli have compatibility ofAvailable in version 2.4.26 and laterseems to be a good idea enable brotli compression.From https://www.brotli.pro/enable-brotli/apache/, the minimum steps would be:
a2enmod brotlias rootAddOutputFilterByType BROTLI_COMPRESS text/html text/plain text/xml text/css text/javascript application/javascripton the Apache VirtualHost of hxldash.comOne thing we could do is take at least the home page of hxldash.com and one or two dashboards (maybe the common case, and maybe one with lots of data that could be compressed, like the geojsons) and inspect the before/after on the pagesize.
Since is possible to force fake request headers, this could be done after brotli already was enabled.
PS.: I can confirm that the https://proxy.hxlstandard.org/data/source does not have brotli enabled (but the response header says
server: nginx, so as commented here https://github.com/SimonbJohnson/quickX3/issues/66#issuecomment-647020995 it may be less simple to suggest implementation there)