uMatrix fails to update MVPS hosts

[najjara]

uMatrix tries to pull the MVPS file, but fails, and falls back to the bundled file (from 2015)

I've noticed this issue on uMatrix only, uBlock pulls it without a problem.

To make uMatrix update the file, I need to open the file's URL, then try to update (which it does successfully).

How to reproduce:

1. Purge cache
2. Update
3. MVPS HOSTS has an outdated triangle sign next to it
4. Click on the chrome:// URL of the file to view it. It's from October-22-2015
5. Go to winhelp2002.mvps.org/hosts.txt, then update again
6. Now MVPS is up to date

uMatrix 0.9.3.6, FF 52.0.2, Arch Linux amd64

[najjara]

How can I show that? I don't want to just take a series of screenshot

[gorhill]

You need to investigate why the network request does not complete on your side. I see no such investigation on your side. Did you look in the logger? Did you try with no other extension installed? Is there anything else in your environment which may cause the issue? Was it just a transient issue with the server? Etc. You will need to investigate this yourself, I can't reproduce, and so far you are the only one reporting such issue.

[najjara]

"Behind the scene" in logger shows that the file is requested, so does Wireshark. I've tried without other extensions and investigated, and found no cause, thus this bug report.

Here's the Wireshark capture: uMatrix-MVPS.pcapng.gz

In it you can see that the first HTTP GET request gets a Bad request response (purged uMatrix, then tried to update). Then a few seconds later I went to the URL where the file resides (this time Firefox sent the HTTP request) to which the stream ends with a HTTP OK message. A few seconds after, I tried to update the host file from within uMatrix, it doesn't send an HTTP request, likely because it checks and finds the file unchanged in FF cache and just parses it (which can be confirmed by purging the cache in uMatrix w/o clearing the FF cache, then trying to update again successfully).

It's not a transient issue, I've been experiencing it for months, but I just did my "workaround" and went about my business. I've decided to report it since I couldn't find any other reports and no fix.

[gorhill]

This is the request headers sent on my side:

Notice the Accept header: */*.

On your side, it seems your Accept header is text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8.

Can you confirm with the browser console as I did above?

[najjara]

Yes, my browser's Accept is text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 (I use the Mozilla binaries, not my Arch package). But it shows the txt file just fine, and uMatrix updates after that without a problem. In Wireshark, I can see that uMatrix sends */*, but then gets a 400 response

Apply tcp.stream eq 0 in filters to follow the first HTTP stream uMatrix makes (that results in failure); it has Accept: */*. But see tcp.stream eq 2 (the browser, which has Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8) but ends in 200 OK

[xofe]

I can reproduce this when using the Random Agent Spoofer extension, with the Headers -> Spoof If-None-Match (ETags) option enabled. Perhaps you have something similar modifying request headers?

[najjara]

@xofe It's good that you've pointed that out. I'm running my network behind Privoxy, and the config is set to filter ETags. I can't see the issue after removing the filter. Weird that only the ASP.NET run server had issues with it.