Possible issue with redirection? (HTTP 302)

[ghost]

Opening the following url in a web browser shows a small PNG file as expected:

https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png

Trying to download the png with the following code will fail.

QPixmap pixmap; auto b = NetworkRequestLoader::loadSync("https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png"); pixmap.loadFromData(b); qDebug() << pixmap.save("D:/Tile.png");

(I tried also the more up-to-date code inside KITScenarist/src/libs/webloader/ with no luck)

The library is redirected instead to an error page (HTTP 404):

https://www.bvg.de/de/System/Ooops-Verfahren

Could you please help to understand what is going wrong? Thank you very much.

[dimkanovikov]

Hello @JuttaAnders,

Thanks for your report, I will check it ASAP. And update repo too with all new library changes.

[dimkanovikov]

Okay, I check it.

Look like you miss the ssl library. Copy libeay32.dll and ssleay32.dll to your executable's folder and voila, data start to load.

[ghost]

Hello @dimkanovikov, thank you for checking.

I'm a bit lost, because the SSL Libraries are there.

Still, my code gets redirected to the error page

https://www.bvg.de/de/System/Ooops-Verfahren

which by the way is HTTPS too!

The QPixmap fails because the content of the QByteArray returned by loadSync is not a image, but the HTML text of the error page.

Replacing the URL of loadSync with the another (HTTPS) URL I get the expected result (an image):

https://a.tiles.mapbox.com/v4/mapbox.satellite/7/66/44.png?access_token=pk.eyJ1IjoibWFwYm94IiwiYSI6ImNpejY4M29iazA2Z2gycXA4N2pmbDZmangifQ.-g_vE53SD2WrJ6tFX7QHmA

I really don't know what is wrong with the https://fahrinfo.bvg.de/ server.

I've also tried to cheat the User-Agent of the library providing the one of the latest version of Microsoft Edge and set the referrer to the home page https://www.bvg.de where the image is supposed to be called from.

Still no success :-(

[ghost]

I have analyzed the data traffic through a proxy (Fiddler) and this is the result with the WebLoader library:

#   Result  Protocol    Host    URL
1   200 HTTP    Tunnel to   fahrinfo.bvg.de:443
2   302 HTTPS   fahrinfo.bvg.de /tiles/base/16/35202/44041.png
3   200 HTTP    Tunnel to   www.bvg.de:443
4   404 HTTPS   www.bvg.de  /de/System/Ooops-Verfahren

Here can we see that the library gets redirected to the 404 Error page (steps 3 and 4).

Raw request sent in step 2:

GET https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134
Content-Type: application/x-www-form-urlencoded
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,*
Host: fahrinfo.bvg.de

Raw answer got in step 2:

HTTP/1.1 302 Found
Date: Fri, 28 Sep 2018 12:23:05 GMT
Server: Apache
Strict-Transport-Security: max-age=10886400; includeSubDomains
Location: https://www.bvg.de/de/System/Ooops-Verfahren
Cache-Control: max-age=20
Expires: Fri, 28 Sep 2018 12:23:25 GMT
Content-Length: 228
Keep-Alive: timeout=5, max=5
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

[ghost]

Here the same request done with Microsoft Edge (just typing https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png in the address bar) :

#   Result  Protocol    Host    URL
1   200 HTTP    Tunnel to   fahrinfo.bvg.de:443
2   200 HTTPS   fahrinfo.bvg.de /tiles/base/16/35202/44041.png

Here there is no redirection.

Raw request sent in step 2:

GET https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134
Accept-Language: en-GB
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Upgrade-Insecure-Requests: 1
Accept-Encoding: gzip, deflate, br
Host: fahrinfo.bvg.de
DNT: 1
Connection: Keep-Alive
Cache-Control: no-cache

Raw answer got in step 2:

HTTP/1.1 200 OK
Date: Fri, 28 Sep 2018 12:22:01 GMT
Server: Apache
Strict-Transport-Security: max-age=10886400; includeSubDomains
Last-Modified: Mon, 07 May 2018 08:17:43 GMT
ETag: "563c-56b994dccf7c0"
Accept-Ranges: bytes
Content-Length: 22076
Cache-Control: max-age=6048000
Expires: Fri, 07 Dec 2018 12:22:01 GMT
Access-Control-Allow-Origin: *.bvg.de
Keep-Alive: timeout=5, max=5
Connection: Keep-Alive
Content-Type: image/png

[dimkanovikov]

Hm... I think redirect can be related to some missed headers. Did you try to imitate browser headers in WebLoader app?

[ghost]

I am closing this issue because it has nothing to do with your code. Using QNetworkAccessManager directly fails the same way. With any kind of combination of browser headers.

qnam->get(QNetworkRequest(QUrl("https://fahrinfo.bvg.de/tiles/base/16/35202/44041.png")));

Either it is a bug of Qt itself, or something is wrong with my OpenSSL installation.

I've tried with the curl library and there is no problem (the version of curl for Windows does not relay on OpenSSL as far as I understand).

Thanks for your help anyway! :-)

[dimkanovikov]

Hm, okay. Glad that you find a way to solve your issue :)