search

claustromaniac / httpz

Fat-free hardenable opportunistic encryption for Firefox
https://addons.mozilla.org/firefox/addon/httpz/
GNU General Public License v3.0
61 stars 5 forks source link

đź”’ Problems with specific sites #2

Closed claustromaniac closed 5 years ago

claustromaniac commented 5 years ago

:exclamation: This issue is locked. Open a new issue instead.

Since version 0.7.0, HTTPZ no longer checks the type of error thrown to decide what to do. Site-specific issues are now rare and they no longer have a common cause, so I prefer to address them separately on a case-by-case basis.

Please use this issue to report any site-specific errors that you come across. Firefox APIs throw a limited number of errors, but this extension won't always handle them all, mostly because

  1. I'm only human and I can (and do) miss some.
  2. Mozilla can add new errors over time, and according to documentation the error messages are not guaranteed to stay the same as the browser is updated.

Make sure to include the URL of the site you're trying to visit so I can try to reproduce your issue on my end and fix it.

ghost commented 5 years ago

Hi @claustromaniac

HTTPZ 0.5.2 still doesn't handle one's localhost (http://localhost/*)

crssi commented 5 years ago

I have a problem with some internal pages, not accessible from Internet, so I cannot give those for example, So I took a time with DDG to hunt a similar behavior web sites. This are the challenges I managed to find:

http://www.pdfcalendar.com/ http://plainfolkcafe.com/

Cheers

claustromaniac commented 5 years ago

@StanGets Sorry, I forgot about that. EDIT: your issue may already be solved in 0.5.3. If that is the case, just ignore the rest of this comment.

I obviously can't reproduce your issue on my end, so I will need you to

@crssi Thanks bud :+1: I will fix those ASAP. You can follow the same steps to make sure you get the same errors in your internal pages. If that is not the case, let me know.

crssi commented 5 years ago

Internal pages gets Error info: NS_ERROR_CONNECTION_REFUSED Thanx

crssi commented 5 years ago

0.5.3 seem much better, but from user experience perspective is a bit slow (takes too long) and before switching to HTTP version it shows sometimes the following:

Insecure Connection Log in to network Server Not Found Invalid URL Blocked Page
Oops.
Log in to network
Hmm. We’re having trouble finding that site.
File not found
Access to the file was denied
Hmm. That address doesn’t look right.
The address wasn’t understood
Unable to connect
The connection has timed out
The page isn’t redirecting properly
Unexpected response from server
The connection was reset
Document Expired
Offline mode
The connection was interrupted
This address is restricted
Unable to find the proxy server
The proxy server is refusing connections
Content Encoding Error
Unsafe File Type
Secure Connection Failed
Your connection is not secure
Blocked by Content Security Policy
Remote XUL
Corrupted Content Error
Unable to Connect Securely
Your connection is not secure
Blocked Page

Firefox can’t load this page for some reason.

You must log in to this network before you can access the Internet.
If that address is correct, here are three other things you can try:

    Try again later.
    Check your network connection.
    If you are connected but behind a firewall, check that Firefox has permission to access the Web.

    Check the file name for capitalization or other typing errors.
    Check to see if the file was moved, renamed or deleted.

    It may have been removed, moved, or file permissions may be preventing access.

    You might need to install other software to open this address.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

    This problem can sometimes be caused by disabling or refusing to accept cookies.

    Check to make sure your system has the Personal Security Manager installed.
    This might be due to a non-standard configuration on the server.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

The requested document is not available in Firefox’s cache.

    As a security precaution, Firefox does not automatically re-request sensitive documents.
    Click Try Again to re-request the document from the website.

    Press "Try Again" to switch to online mode and reload the page.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

    Check the proxy settings to make sure that they are correct.
    Check to make sure your computer has a working network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

    Check the proxy settings to make sure that they are correct.
    Contact your network administrator to make sure the proxy server is working.

    Please contact the website owners to inform them of this problem.

    Please contact the website owners to inform them of this problem.

    The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
    Please contact the website owners to inform them of this problem.

The owner of has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website.

Firefox prevented this page from loading in this way because the page has a content security policy that disallows it.

    Please contact the website owners to inform them of this problem.

The page you are trying to view cannot be shown because an error in the data transmission was detected.

    Please contact the website owners to inform them of this problem.

Advanced info: SSL_ERROR_UNSUPPORTED_VERSION

uses security technology that is outdated and vulnerable to attack. An attacker could easily reveal information which you thought to be safe. The website administrator will need to fix the server first before you can visit the site.

Error code: NS_ERROR_NET_INADEQUATE_SECURITY

Firefox did not connect to because your computer’s clock appears to show the wrong time and this is preventing a secure connection.

Your computer is set to , when it should be . To fix this problem, change your date and time settings to match the correct time.

Firefox did not connect to because your computer’s clock appears to show the wrong time and this is preventing a secure connection.

Your computer is set to . To fix this problem, change your date and time settings to match the correct time.

Learn more…

Report errors like this to help Mozilla identify and block malicious sites

It looks like your network security settings might be causing this. Do you want the default settings to be restored?

sometimes:

Unable to connect

Firefox can’t establish a connection to the server at servername.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.
ghost commented 5 years ago

@claustromaniac , HTTPZ updated to 0.5.3 and the results are:

1- Access to my http://localhost/* pages now performs correctly, though the page, the one which happens to be my Firefox Homepage/NewTab page, is sometimes preceded by a white flash... odd. Seems random, hard to explain. Random because happened once, then again after FF restart with cache cleaned. I know I'm badly informing, I'll have to try to find a reproducible scheme.

2- Problem with an http-only site (http://www.cartesfrance.fr/) which remains stuck on https. The console gathered this:

Error: WebExtension context not found! ExtensionParent.jsm:1039:13 cannot inject content script /js/bootstrap.js into url https://www.cartesfrance.fr/ on tab 6 and frame 0 utils.js:188:7 cannot inject content script /js/contentscripts/fingercounting.js into url https://www.cartesfrance.fr/ on tab 6 and frame 0

No problem with https sites accessed via http : they seem to be all handled correctly.

NOTE: http://www.cartesfrance.fr/ seems to be an exception, I've tested other http-only sites which get flawlessly reverted to http (http://browserspy.dk/, http://www.toolsvoid.com/, http://www.guichetdusavoir.org/ for instance).

claustromaniac commented 5 years ago

@crssi

is a bit slow (takes too long)

That happens when the request over HTTPS results in a timeout, as was the case for http://www.yuntrack.com/. It should happen only once per session, since the extension will ignore requests to that host afterwards. Maybe I can do something more about that, though..

Leaving timeout errors aside, it is expected that a first try over HTTPS + a retry over HTTP will always take longer than making a simple request over HTTP directly.

As for the super long list of errors displayed on screen, that is merely a cosmetic issue that happens because the extension doesn't let Firefox finish loading the error page. It interrupts the loading of that page because it immediately retries the request over HTTP. I didn't see any reason to let Firefox finish loading error pages, it's just wasted time.

@StanGets White flashes are expected. They occur for the same reason I just explained in this very comment: the extension doesn't let Firefox finish loading error pages. It's only a cosmetic issue and should happen only once per host per session.

As for the issue with https://www.cartesfrance.fr/, I'm afraid I can't reproduce it. I will need you to follow the steps in this comment, because the console output you pasted is not related to this extension.

claustromaniac commented 5 years ago

Hm. I'm thinking the extension should instead just ignore all requests to localhost, as in, not even try over HTTPS. EDIT: done 5b60fc6

ghost commented 5 years ago

@claustromaniac hereafter the console's error display after visiting https://www.cartesfrance.fr/ :

Navigation error tabId: 5 url: https://www.cartesfrance.fr/ error: Error code 2153394167

I had processed as mentioned in your comment but missed the first line, sorry.

Hm. I'm thinking the extension should instead just ignore all requests to localhost, as in, not even try over HTTPS.

I think that would be OK given localhost is never accessed via https as far as i know, anyway not here.

ghost commented 5 years ago

@claustromaniac that's exactly what I had done but again I see no specific Error info

Screenshot of my Browser Console after 1- Cleanup (trash on the left), 2- Calling http://www.cartesfrance.fr/

greenshot-2019-01-08-100859

I'm missing something...

claustromaniac commented 5 years ago

In that screenshot you have all filters selected except Info (which is the only filter you need to select). :sweat_smile:

ghost commented 5 years ago

Upside down Browser Console logic : I see 8 buttons, I choose info and that means I've de-selected info?

Anyway, as far as i'm concerned I'll stick on HTTPSEverywhere and its feature which enables adding rules : when I encounter an http site I toggle it to https (via a dedicated basic extension button) and if it's ok I add a rule for that site. IMO the best of two worlds, that of the sig/list together with the liberty of adding site specific rules. I think I'll stick on that approach for now.

Good luck with HTTPZ :=)

EDIT: for the sake of remaining civilized and coherent (lol!) I've again tested http://www.cartesfrance.fr/ with HTTPZ 0.5.3 and the console (this time correctly handled, ouf) displayed this:

Error info: Peer using unsupported version of security protocol. httpz.js:78:7

Hope that helps. I admire coders', programmers' patience and determination, their talent. All I lack :=)

claustromaniac commented 5 years ago

IMO the best of two worlds

Except that you need to make an insecure request first (that anyone inspecting the traffic can see clearly).

Thanks for your feedback so far.

ghost commented 5 years ago

Except that you need to make an insecure request first (that anyone inspecting the traffic can see clearly).

Yeah, that's the point which haunts me in that approach. Which is why I was and remain interested by the systematic https try-it-first scheme. I can hide myself but that eye sees me :=)

Anyway, @claustromaniac no problem here in testing HTTPZ, if there's anything I can do to help, it'll be my pleasure, especially for you, I mean someone dedicated to elaborating smart code and extensions. It's only that my default approach the time being remains HTTPSEverywhere. But I may change advice. Never say never, nor always.

claustromaniac commented 5 years ago

EDIT: for the sake of remaining civilized and coherent (lol!) I've again tested http://www.cartesfrance.fr/ with HTTPZ 0.5.3 and the console (this time correctly handled, ouf) displayed this:

I appreciate that. That's very helpful. :+1:

ghost commented 5 years ago

I'll keep an eye on HTTPZ updates and if I find anything I believe may be helpful of course I'll share it here. I remain tuned on, no problem. It's only that i've had a hard day (I'm not the only one, I know) and then I occasionally become hot, I mean excessively reactive :=)

Carry on the good work, @claustromaniac , I know you will.

claustromaniac commented 5 years ago

No problem! Thank you for your interest, for following my work, and for your continued feedback. It's more than what I normally expect to receive in return, and it is both helpful and motivating. :heart:

ghost commented 5 years ago

Just installed HTTPZ 0.5.4 (and of course disabled HTTPSEverywhere).

What will be more bothering is that HTTPZ keeps the log for the session only which means that that white page (https to http on first run) will appear at every new session. If anything could be done about that (as my boss used to say!) life would be easier!

claustromaniac commented 5 years ago

What will be more bothering is that HTTPZ keeps the log for the session only which means that that white page (https to http on first run) will appear at every new session.

One thing I've been considering is to allow users to extend the period to X number of days or something like that.

ghost commented 5 years ago

One thing I've been considering is to allow users to extend the period to X number of days or something like that.

I think the session period is pertinent, I have in mind rather a way to prevent this white page display to start with. I'm aware every extension is different yet when they deal with the same objective a comparison is maybe allowed : look at HTTP to HTTPS and Smart HTTPS, neither suffer from this white pre-load page when reverting to http. Isn't this feasible with HTTPZ?

crssi commented 5 years ago

^^Its ugly and for some people confusing (seeing a screen of errors in between), but at least it works. Smart HTTPS does not work together with TC. HTTP to HTTPS breaks when you click on Aliexpress orders track shipment.

But HTTPZ works in both cases as it should. At least for now (0.5.4) I haven't found any breakage.

ghost commented 5 years ago

Smart HTTPS does not work together with TC. HTTP to HTTPS breaks when you click on Aliexpress orders track shipment.

Thanks for the info, @crssi . I use neither TC not Allexpress so I learn.

Nevertheless we'd have to consider a relationship between SmartHTTPS' failure, HTTP-to-HTTPS' failure and their ability to avoid the 'white page' to abandon the idea HTTPZ could avoid that 'white page' which is really a visual nuisance, IMO, even within a session period, even within whatever longer period.

claustromaniac commented 5 years ago

look at HTTP to HTTPS and Smart HTTPS, neither suffer from this white pre-load page when reverting to http.

https://github.com/ilGur1132/Smart-HTTPS/issues/23

I've never used either of those, but they seem to do a bunch things I don't like.

I can let Firefox finish loading the error page, so it doesn't look like a "white flash", but I don't think that will make much of a difference (most error pages are white by default anyway). Also, that would be slower. Other than that, I could try to replace the error page with a custom page of my own, but there are too many good reasons not to do that, so I won't. Another possibility is to interrupt the navigation (using the navigation API) and fetch the HTML document in the background before making the actual request, but that will have the drawback that when sites do support HTTPS, the extension will still make that second unnecessary request, that can even be used to track you (because fetching the HTML doc twice every single time it is not a standard behavior in any way). Moreover, all of these routes would require adding a significant amount of code and complexity just for that purpose.

I will fix that cosmetic issue if I find a reasonable way to do it, you can count on that. However, I doubt that will happen, considering we can do very limited stuff with web extension APIs.

For now the "white page" will continue to be a known issue. I'm sorry.

ghost commented 5 years ago

I understand, @claustromaniac . At this point it appears that the approach combining efficiency (done with HTTPS 0.5.4) with optimal security seems to leave no alternative to accepting this "white preload page" (even if you evoke in perspective a reasonable way to do it).

Maybe the concept of try https and revert to http if failure is just not reasonable at this time given the "very limited stuff with web extension APIs", unless to consider security before cosmetic (and time gap even if short) issues.

I have in mind this option, which would require extra code of course but I guess (I "guess") not excessively tough, meaning an accessible whitelist:

The http-only log at this time is session only. Maybe adding in HTTPZ these options:

Keep log for
1- Session only (default) -- set and forget
2- User's choice (days) -- set and forget
3- Always [as in SmartHTTPS] -- set but erase the log once in a while

Log : clean now

This way users who never look at the options (many!) would remain protected and those who wish to challenge cosmetics to security (count me in!) would be at ease to combine both given their ratio of security/cosmetics ...

Just my 2 cents.

claustromaniac commented 5 years ago

Yes, that is what I had in mind when I said...

One thing I've been considering is to allow users to extend the period to X number of days or something like that.

I would like this feature myself, so I was already kind of planning to implement it (even before you brought up the "white page" issue).

EDIT: Done.

Maryse47 commented 5 years ago

Enabling this addon causes connection to 192.168.0.1 (which doesn't support https) to fail (timeout). Perhaps all connections to LAN should be whitelisted similarly to localhost.

Also how to handle sites which theoretically have https version but it's not functional yet?

claustromaniac commented 5 years ago

Thanks for reporting that. 0.6.2b should fix it. It's a minimal change so you don't need to test it if you don't want to. I'll do the release on AMO later.

Also how to handle sites which theoretically have https version but it's not functional yet?

I'm not sure I understand what you mean. You can always "whitelist" sites since 0.6.0 if you notice something wrong with them (by clicking the address bar icon).

Maryse47 commented 5 years ago

I'm not sure I understand what you mean. You can always "whitelist" sites since 0.6.0 if you notice something wrong with them (by clicking the address bar icon).

That's right, sorry.

ghost commented 5 years ago

HTTPZ 0.6.2, now still as beta here though an extra-minimal change, will be updated when available on AMO. Pity for users who know nothing of GitHub, rely only on AMO, and install 0.6.0 or don't update 0.6.0 whilst the update is now available, but only here. Minimal update maybe but valuable : think of users who encounter LAN connections issues with HTTPZ 0.6.0 and who might uninstall for that sole reason, a pity it would be.

EDIT: HTTPZ 0.6.2 now available on AMO (if I had just been patient enough!).

ghost commented 5 years ago

Hi,

HTTPZ 0.6.2 / Firefox 64.0.2 (x64) / Windows 7 (x64).

I'm encountering a strange issue with https://www.tomshardware.com/ The page is not displayed but only this message: An error (403 Forbidden) has occurred in response to this request.

The page displays correctly if HTTPZ is disabled.

EDIT: confusion

Even with HTTPZ disabled the page https://www.tomshardware.com/ displays the error message. My mistake because I had tested that page with HTTPS Everywhere instead of HTTPZ this very morning and the page had displayed correctly (I test a lot).

I believe https://www.tomshardware.com/ has a problem of its own, and when it does display it appears chaotically as if javascript was blocked. Something nasty with that site.

Please forget this comment.

crssi commented 5 years ago

Just wanted to mention that I have tested with my tightened profile and with a brand new profile and everything works a it should here. I am sure that www.tomshardware.com suffered its own intermediate problem at time you noticed "breakage", which is now already sorted out. But for sure, that page is a super nasty one.

ghost commented 5 years ago

I've tested again, 5 minutes ago, this https://www.tomshardware.com/ and nothing does it, always the An error (403 Forbidden) has occurred in response to this request. error message.

Tested with HTTPZ enabled and disabled, same issue: nothing to do with HTTPZ Obviously related to another extension and/or an about:config setting.

Sorry for the disturbance, the issue doesn't belong here. @crssi , the site is displayed correctly on your system so it's really my configuration.

Gitoffthelawn commented 5 years ago

Here's a site that is currently incompatible with HTTPZ: http://amlpages.com/about.shtml

Maybe issue #6 does make sense after all?

Gitoffthelawn commented 5 years ago

Here is a site that has a different compatibility issue with HTTPZ: http://www.dotnetvishal.com/

Here's another one with the same issue: http://www.android-x86.org/

Gitoffthelawn commented 5 years ago

I tested 0.7.1 on the sites I listed above, and it works great. Thank you!

claustromaniac commented 5 years ago

Thank you all for the feedback :cat:

Madis0 commented 5 years ago

http://dti.tlu.ee/digiois/?lang=en will load indefinitely on HTTPZ 0.7.2

claustromaniac commented 5 years ago

@Madis0,

I can't reproduce that. For me, the request times out after around 20 seconds, and then HTTPZ starts a request over HTTP that completes succesfully (as it should).

If it really keeps loading over 20 seconds for you, please, follow these steps:

  1. Open the console (CTRL+SHIFT+J).
  2. Go to about:addons and disable the extension, then enable it again.
  3. Go to http://dti.tlu.ee/digiois/?lang=en
  4. See if HTTPZ: something appears in the console output after a while.
  5. Copy and paste here.
Madis0 commented 5 years ago 
HTTPZ: settings loaded
HTTPZ: NS_ERROR_NET_TIMEOUT
HTTPZ: local storage changed

You're right, it's not actually indefinite but 20 seconds is quite long to wait for anyway. Could the user configure that delay?

claustromaniac commented 5 years ago

Could the user configure that delay?

See #7. I certainly can add that functionality, but it would introduce compatibility issues, not to mention that it would take a rework of the extension because it was designed from the ground up with compatibility in mind.

I had a different idea that might be worth trying out, though. The extension could instead show an internal page saying something like "redirecting to https://www.example.com" with a button like "It's taking too long!" that aborts that action and takes you to http://www.example.com.

practik commented 5 years ago

http://living.corriere.it goes into a death spiral (constantly reloading) when HTTPZ is enabled. Lots of mixed content errors in the console. I'm not sure if there's anything to be done, other than whitelist the site. (Also not sure if I should have opened a new issue for this, but this seemed to be the right place.) In any case, thanks for the extension!