Not compressing anything

[Hankrecords]

Hello. I'm going crazy here: I've been struggling with compression (compression in general, not just your plugin "Compression" in particular) for a couple days, because I don't seem to be able to make any compression work.

I tried with Compression first, and it didn't work, so I tried zlib, gzippo and gzipStatic, to no avail. I know that this probably means that it's not a problem with Compression but it has something to do with my project, but any help would be so much welcome.

In the request headers, there's accept:gzip, the response though is full size and doesn't have any content-encoding header. I tried setting DEBUG=compression, and it does log compressions (ranging from 10ms to 400ms) but it doesn't seem to be working.

I tried setting the file size threshold to zero (although it wasn't working on a 1MB file either) and to replace the filter function with a "return true", and also tried setting NODE_ENV=production. None of this worked. Also, I put app.use(compression({options})) as high as possible in my code.

See my question on StackOverflow for pictures, code and a little more details

UPDATE: I tried making a clean new project to try just compression. This is the code:

const   express = require('express'),
        fs = require('fs'),
        compression = require('compression');

var app = express();

app.use(compression({threshold:0, filter:function(){return true;}}));

app.get("/test", function(req,res){
    fs.readFile("./test.js", function(err, content){
        if(err) {
            console.log("errore");
            var h = {'Content-Type':'text/plain'};
            res.writeHead(200, h);
            res.end("ERR", "utf-8");
        }
        else {
            var h = {'Content-Type':'application/javascript'};
            res.writeHead(200,h);
            res.end(content);
        }
    });
});

var server = app.listen(6969, function(){
   var host = server.address().address;
   var port = server.address().port;

   console.log("\n" + host + ':' +port + " - " + (new Date()).getHours() + ":" + (new Date()).getMinutes() + "\n");
});

These are the headers:

DEBUG says compression is actually compressing:

(at first it said size below threshold, although it was angular.js which is above 1MB and the default threshold is 1KB... anyway, I set the threshold to zero now and it doesn't say that anymore)

But as you can see by the headers, nothing gets compressed.

UPDATE 2:

Now this is getting weird. I installed Curl to try a request, and it works with XGET:

But it doesn't work with regular GET:

The browser apparently sends a different type of GET, it only works with curl. What does this mean? How do I fix this?

[Hankrecords]

Apparently, https://checkgzipcompression.com/ says that my website is indeed compressing.

I just don't seem to be able to see the Content-Encoding header anywhere: I've tried Fiddler, Chrome, Firefox Dev... the only thing that gets it gzipped properly is curl. I also tried doing it from a different computer on the same network. Might it be that it only gets gzipped when the request comes from outside the server's network? I'm going to try that now.

[dougwilson]

Hi @Hankrecords I have never heard or seen of such behavior before. I'll try to setup your code and reproduce the issue. If you're able to determine the cause before I get to it, please absolutely feel free to submit over a PR with any fix :) !

[Hankrecords]

Hi @dougwilson, I've made a few other tests and I can confirm that it works perfectly when the request is made from outside my network, while it doesn't get gzipped when I load the website from within the network, regardless of which pc I'm using. This is incredibly weird to me... I mean, I would have expected the network proxy to interfere with something (given that I'm on my company's network and not a home network) but that would mean having problems when contacting the server from outside, not the other way around. So I doubt that the network's the problem. It's not even a problem with the request headers AFAIK, since the Accept-Encoding:gzip is always there.

[dougwilson]

Using your app, I have not been able to reproduce your issue, so I'm not sure what (if anything) I can change with the module. Any hints you can provide?

[Jezternz]

Not sure if I am seeing the same problem also. But I have noticed when accessing my files running express locally I get this:

When accessing them online (ec2 instance/amazon) I get this:

The results of the checkyourcompression are also confusing for me, I get:

Really not sure where to turn next. I also added a filter function like this:

app.use(compression({ filter: (req, res) => { var x = compression.filter(req, res); console.log('to-be-compressed', x, ' ', req.originalUrl); return x; } }));

Interestingly the file gets a to-be-compressed: true. As if it is being compressed even when I can't see that in chrome dev tools.

Not sure if relevant, but I also tried serving gzip files built in my build process using connect-gzip-static module, but that returned exactly the same results (compressed locally, uncompressed online).

[dougwilson]

Hi @Jezternz since your resource is a public URL, I was able to load it. Here is a screenshot of that URL loaded in my Chrome, showing compressed:

[Jezternz]

@dougwilson - maybe something funny is going on, like caching somewhere between me and amazon, maybe ISP caching it or something? I have triple checked and my client definitely isn't caching. Maybe I will leave it a couple of days and revisit. Thanks for the heads up.

[Hankrecords]

@Jezternz Apparently we have the exact opposite problem hahaha I still haven't figured out what might be happening.

But I've just tried to visit my live website (and yours) from the PC I'm working on (and which doesn't get gzipped data) and... ta-da! It isn't getting gzipped from the live versions either.

I'm starting to think it's just something with the client. I'm on Chrome v. 56.0.2924.87, what about you guys?

[dougwilson]

@Hankrecords I am using Chrome Version 56.0.2924.87 (64-bit)

[Hankrecords]

@dougwilson Ok, so it doesn't have anything to do with Chrome's version. Is there any chance my antivirus (ESET) could be interfering with the gzip only when the request comes from within the network? (I highly doubt that, it wouldn't make any sense)

[dougwilson]

@Hankrecords I'm not sure, but I assume if you can disable it, you could determine that. I would say does the difference you see is actually http vs https sites (not internal vs external)? Because http, since it is not encrypted, can be intercepted and altered between you and the origin server by proxies and programs, potentially stripping any compression in the process.

[Jezternz]

Disabling ESET for me made no difference.

I think you may be on to something regarding the https though. I have noticed when visiting a few sites, I can't seem to get GZIP working for HTTP, but will work for HTTPS. @Hankrecords try the economist, for me the results were: http://www.economist.com/ NO GZIP https://www.economist.com/ GZIP

Might be time I look into getting an https cert... to improve site speed haha O.o

I don't suppose anyone here would have any ideas of how you could work out where this is going wrong? - I am with the biggest ISP in Australia, so if the problem lies in their chain; would like to inform them if they happen to be slowing down the internet for a country. Edit: Not ISP, confirmed with another user.

[dougwilson]

Unless there is something obvious, the best way would be process of elimination. For example, if you move your computer to a completely different network, does it persist? If so, seems like something on the computer. If you take a computer without the issue to the same network your computer is on and it doesn't happen to the new computer, then that would also help narrow it down to the computer.

[Hankrecords]

@Jezternz I can tell you that my problem isn't with HTTPS, since the live version of my website is on HTTP (it's still under construction and I don't have a certificate yet) and gzip works there. @dougwilson I'm on a rather large company network, so moving the pc to another network isn't really a viable option :/

[dougwilson]

Any suggestions to what the issue is would help a lot. Since I have not been able to reproduce, I'm not sure how I can help here.

[Hankrecords]

At this point I don't think it's a problem with your module, @dougwilson

[dougwilson]

Ok, I'm going to close it for now, but if there is something I can change or do, please let me know!

[black]

Anyone please, even I am getting the same issue I read the whole thread and tried everything that has been suggested here and nothing happened. I don't know what's happening. First, I thought running the local express app in incognito mode would be the problem but it doesn't work in normal window either. (Chrome) No compression whatsoever.