Protocol issues

[marcselman]

Hi,

I have the following URL which does not work: https://images.weserv.nl/?url=www.that-is-it.nl%2Fwp-content%2Fuploads%2F2018%2F07%2FHoomall-Honden-Mat-Draagbare-Reizen-Waterdichte-Gemakkelijk-Schoonmaken-non-stick-haar-Katten-Slaap-Matten-Pad-Hond.jpg&w=75&h=65 Now if I add // (%2F%2F) in front of the domain is does work: https://images.weserv.nl/?url=%2F%2Fwww.that-is-it.nl%2Fwp-content%2Fuploads%2F2018%2F07%2FHoomall-Honden-Mat-Draagbare-Reizen-Waterdichte-Gemakkelijk-Schoonmaken-non-stick-haar-Katten-Slaap-Matten-Pad-Hond.jpg&w=75&h=65

But then I also have this URL which does work: https://images.weserv.nl/?url=www.boot-tapijt.nl%2F221%2Fsjippie-rubberboot-spuitwax-uv.jpg&w=395&h=300 But if I add // in front of the domain is does not work: https://images.weserv.nl/?url=%2F%2Fwww.boot-tapijt.nl%2F221%2Fsjippie-rubberboot-spuitwax-uv.jpg&w=395&h=300

Your documentation says there should be no protocol or // in front of the domain but for the first image above it solves the problem.

So in some cases it works better with, in other cases better without. Is there a way it always works?

Thanks!

[andrieslouw]

First URL returns 404, because the server isn't responding with content on http, only on https: https://redbot.org/?uri=http%3A%2F%2Fwww.that-is-it.nl%2Fwp-content%2Fuploads%2F2018%2F07%2FHoomall-Honden-Mat-Draagbare-Reizen-Waterdichte-Gemakkelijk-Schoonmaken-non-stick-haar-Katten-Slaap-Matten-Pad-Hond.jpg

This is considered bad behavior. The URL is also not functional in any browser I tested, unless I explicitly add https:// in front. See: http://www.that-is-it.nl/wp-content/uploads/2018/07/Hoomall-Honden-Mat-Draagbare-Reizen-Waterdichte-Gemakkelijk-Schoonmaken-non-stick-haar-Katten-Slaap-Matten-Pad-Hond.jpg

Second URL responds to https-requests with 404, but will work on http: https://redbot.org/?uri=https%3A%2F%2Fwww.boot-tapijt.nl%2F221%2Fsjippie-rubberboot-spuitwax-uv.jpg

Which is also stupid, but a bit less stupid than the first case.

If any site is to support https, they should redirect http-requests. We honor any redirection, be it from http to https, or https to http. However, we are unable to guess the protocol, and our behavior is comparable with most browsers (with the exception of HSTS, which is something we don't support for origin hosts at this time).

And, as you may have guessed; // is rewritten to https://. If any resource is protocol-independent, we prefer https.

You can use ssl: to prefix hosts with https://, if there is no prefix given, we use http://. This design is over 12 years old, because at that time lot's of forum's were having problems with parsing http://images.weserv.nl/?url=http://....

[marcselman]

You already have an errorredirect parameter. Is it an idea to add an extra parameter, something like discoverprotocol that will first try https and if that returns an error, try http and if that returns an error load the errorredirect url?

Or else, allow multiple errorredirect urls semi-colon separated or something?

[neuotq]

HI there. I have similar problem. Here it the image url: https://statuskrym.ru/storage/buildings/2OqywbIi2Cb9OBh8wm8olwo6CYZLrGKs8VrMtfpq.jpeg There is redirect from http to https red bot http redbot https

But I get 404 error with error message "Operation timed out."

https://images.weserv.nl/?url=ssl:statuskrym.ru/storage/buildings/2OqywbIi2Cb9OBh8wm8olwo6CYZLrGKs8VrMtfpq.jpeg and https://images.weserv.nl/?url=statuskrym.ru/storage/buildings/2OqywbIi2Cb9OBh8wm8olwo6CYZLrGKs8VrMtfpq.jpeg

[andrieslouw]

@neuotq : Unfortunately that has to do with Russia government blocking large parts of the internet, including our service providers. See https://blog.online.net/2018/04/20/important-note-about-the-ip-address-ranges-blocked-by-russia/ for our ranges. I cannot communicate with the IP-address behind statuskrym.ru due to it. It's the Russian version of the Chinese firewall, only even worse (as it's blocking completely, instead of monitoring and throttling).

[neuotq]

thnx for reply. Forgot about ip blocking .. meh. Also im from Crimea, and we (common users) here have bingo: blocks from our government + sanctions from USA gov, so many web services dosent work. Anyway thanks, I will think how to deal with that.

[andrieslouw]

@neuotq : Politics and internet seem to be an increasingly bad combination. I hope you'll be able to use VPN's or Tor. Cloudflare is increasing the support for Tor, maybe that will make some difference. However, I'm worried about Cloudflare being an U.S. company, and the U.S. also being in a really bad political climate.

I've looked into technologies to use on this service to bypass the Roskomnadzor blocks, but it's an uphill battle, which is hard to keep up with for us (costs, effort). Let's hope Telegram will win this battle, and Russia will change it's mind, as the blocks are increasingly undermining day-to-day operations for everyone involved.

Please let your voice be heard during elections, and motivate others to do so. We're only able to provide the free and open services we'd like to, with the support of others doing the same.

To a better future :clinking_glasses:

[marcselman]

@andrieslouw Could you take a look at my suggestions above? I'm curious if this is something that could be included.

Thanks!

[andrieslouw]

@marcselman : Ah, missed it, yes, we'll look into it, but need to check if this won't cause any unintended behavior. There's probably a reason large browsers operate in the same way as we do.

[andrieslouw]

Note to myself: still need to take a look at the "https fail -> try http" part.

[andrieslouw]

I'm not sure if it is beneficial to implement "HTTPS failed -> try HTTP" because of the possibility of downgrade attacks. We may want to add HSTS-support though, which would accomplish the same thing, but in a different direction (try HTTP, but upgrade connection if possible).

[kleisauke]

Note that HSTS Preloading should already be supported, since we automatically follow redirects. It's mandatory to create an explicit redirect from HTTP to HTTPS when you submit your website to the HSTS Preload List.

In order to be accepted to the HSTS preload list through this form, your site must satisfy the following set of requirements:

1. Serve a valid certificate.
2. Redirect from HTTP to HTTPS on the same host, if you are listening on port 80.
3. Serve all subdomains over HTTPS. (...)
4. Serve an HSTS header on the base domain for HTTPS requests: (...)

I'll close this issue, given that there's no further work to be done here. Upstream servers should be responsible for redirects, a fallback mechanism other than the &default= parameter is not something we consider.