https tracker failure

[9Pigs]

Please provide the following information

qBittorrent version and Operating System

qB ver 4.3.4.1 MacOSX Mojave 10.14.6

If on linux, libtorrent-rasterbar and Qt version

(type here)

What is the problem

torrenting.com trackers fail (not working)

There is a work-around: edit tracker "https:" to "http:"

What is the expected behavior

Trackers work

Steps to reproduce

Any torrent from https://www.torrenting.com Observe "not working". Change the https to http -- and it refreshes and works.

Extra info(if any)

Reverting back to qBittorrent 4.3.3 picks up and works just fine.

[ghost]

Try turning off Validate HTTPs Tracker certificates from advanced settings page.

[9Pigs]

Try turning off Validate HTTPs Tracker certificates from advanced settings page.

That sounds like a workable work-around --- but is blowing off certificates a sensible thing to do? What changed in the programming? Does my using a vpn make it a moot point?

[FranciscoPombal]

The new code is better - now you know which sites are actually using valid certs and can decide for yourself if you should still trust them or not. If a cert fails validation, that's either the site's fault or maybe your OS's CA store is outdated.

[aeular]

qBittorrent version and Operating System

Mac Big Sur, qBittorrent 4.3.4.1

What is the problem

Sites with valid SSL certificates are not working as a tracker

What is the expected behavior

qBittorrent to connect to a tracker with a valid SSL cerficiate when set to https with verify enabled.

Extra info(if any)

This isn't happening only on invalid certificate sites. This is also affecting sites with valid Let's Encrypt certificates, issue specifically happening on Mac (but not windows). Firefox, Chrome, Safari, SSL Labs, DigiCert, qBittorrent windows 4.3.4.1 (with cert validation enabled) all are cool with the certificate, but qBittorrent on Mac Big Sur is not okay with it for the tracker. And yes, it's all running on the same webserver for 0% variance of certificates.

Also, qBittorrent on mac or windows is able to download the .torrent file from the same webserver that the tracker is on. Not sure if you extended verification to there (or if that's handled differently or by OS).

Yes turning off Validate HTTPs Tracker certificates from advanced settings page is a work around. But this is a bug. I can also provide an example with this issue if you want, though privately as it is a private tracker.

A bit of certificate data (don't want to say site name) from DigiCert's validation: Issuer = R3 Key Length = 254 Signature algorithm = SHA256-RSA

OCSP Staple: | Good OCSP Origin: | Good CRL Status: | Not Enabled The certificate expires May 27, 2021 (63 days from today) TLS Certificate is correctly installed

[FranciscoPombal]

@aeular

qBittorrent's HTTPS validation is really libtorrent's. Maybe it has some kind of problem with Big Sur and possible other macOS versions. Ping @arvidn, can you reproduce?

[aeular]

I have a test torrent file that hits a page that returns a bencoded response (not actually a tracker, but works except in this case), that does it on a fresh Big Sur VM, and reports from users of the site of Mac OS 10.14 and 10.15 with the issue too, and that disabling validation is a fix. I can send the torrent file to someone for reproduction if needed. The most obvious thing I can think of is a root trust issue. I can also provide example nginx configuration for let's encrypt certificate that has the problem.

I have also verified it's not related to OCSP stapling. Valid stapling or no stapling does not change outcome.

[arvidn]

libtorrent uses OpenSSL and relies on the system certificate store as identified by OpenSSL. There's a special case on windows where it doesn't keep a certificate store in the same way OpenSSL expects. However, Linux and Mac OS are treated the same.

[arvidn]

@aeular does curl or wget work to authenticate HTTPS? I would expect them to use the same certificates.

[aeular]

@arvidn after a brief fix to fix the VM's dns (aka it suddenly had no servers): curl reports ssl cert fine, and openssl s_client returns valid cert and OCSP on big sur. No wget on it, but those should cover the openssl side. And qBittorrent stil just simply has "not working" for the tracker with no messages to help, but still works as soon as validate https tracker certificates is turned off

[arvidn]

I don't know off-hand which libtorrent version is used in qB ver 4.3.4.1, but I assume it's 1.2.12.

I tried to reproduce this on Mac OS Big Sur by creating a torrent and setting the tracker to https://github.com/announce and I correctly get a 400 Bad Request. The certificate validation appears to work.

As a negative test I also created a torrent with https://untrusted-root.badssl.com/announce as the tracker, which correctly failed certificate validation.

I tested the RC_1_2 branch in git, but I don't think there has been any changes to this logic since 1.2.12.

[aeular]

@arvidn the about shows Qt 5.15.2, Libtorrent 1.2.13.0, Boost 1.75.0, OpenSSL 1.1.1i, and zlib 1.2.11

[aeular]

Oh, interesting, both openssl and curl report LibreSSL 2.8.3, instead of openssl, no clue if that matters in the end.

[aeular]

And I'm correcting myself, qBittorrent is using openssl 1.1.1j, just the font in about makes it really unclear (had to copy it out to be sure, after noticing windows was 1.1.1j)

[9Pigs]

@an0n666:

Try turning off Validate HTTPs Tracker certificates from advanced settings page.

That sounds like a workable work-around --- but is blowing off certificates a sensible thing to do? What changed in the programming? Does my using a vpn make it a moot point?

I looked and my "What changed?" question has an answer ---- the older version default was not to validate, new version default is to validate.

[arvidn]

but do we know what the problem is? If the certificates from the trackers are valid, it should work when validating them.

[aeular]

@arvidn no, I don't know the root, but I might have just discovered an important key issue with the testing. You created a torrent with https://github.com/announce and I decided to try the same. Changed the tracker to that, with verify off, get the expected error message in the field "Bad Request", good, all to plan.

Now let's turn on the verify certificate on, quick reboot to be sure things are clean and: tracker line status: "Not working" with no message on why.

I think we can agree that github has a valid SSL certificate setup right now. So why is this suddenly different?

[arvidn]

I think we can agree that github has a valid SSL certificate setup right now. So why is this suddenly different?

You mean why is it working for me but not for you?

I'm not sure what you mean by "suddenly".

Perhaps one difference might be that I build against Homebrew's openssl, and when I install it, perhaps homebrew also install some root certificates. It sounds a bit far fetched perhaps. But I imagine qBT links against openssl that it then ship in its app bundle, without any root certs.

[arvidn]

actually, it seems like homebrew does ship a root CA bundle with openssl.

https://github.com/raggi/openssl-osx-ca

I'm guessing there needs to be a special case for MacOS, just like for windows, where the system is queries for its root CA certs and explicitly added to the SSL context. Any help doing this would be appreciated. Surely someone else must have done this.

[aeular]

Perhaps one difference might be that I build against Homebrew's openssl, and when I install it, perhaps homebrew also install some root certificates.

Are you building qBittorrent or just libtorrent? I guess I just didn't bother thinking to realize you probably are directly testing libtorrent.

Is there a way for me to find out on mac where it's currently trying to pull the trust store from I wonder.

And maybe just set it to use the same root trust that LIbreSSL that ships with mac uses, which appears to just be /etc/ssl/cert.pem and /etc/ssl/certs/

Alternatively, perhaps just allowing qBittorrent to pass the root trust in via config (also in advanced) to libtorrent, like rtorrent's network.http.capath.set option, to be the most flexible on mac and linux?

[arvidn]

the openssl docs says it will look at the SSL_CERT_DIR and SSL_CERT_FILE environment variables. You could try setting those in the environment you run qBT in. (You probably need to run it from the terminal for that)

[aeular]

Just did a quick test and it is for sure a root trust issue.

export SSL_CERT_FILE=/etc/ssl/cert.pem
export SSL_CERT_DIR=/etc/ssl/certs
open -a qBittorrent

And then retest any of our test cases, and they work properly now.

[FStars]

I can confirm, that 4.3.4.1 works with my private trackers on both MacOS Big Sur and Catalina when turning Try turning off Validate HTTPs Tracker certificates from advanced settings page.

I have checked the certificate. It is valid and works nicely with Deluge and Transmission - latest versions.

[FStars]

I really don't understand why this is a bugfix? To me it means that a lot of users suddenly have a lot problems because of this change in a default setting - and it creates a lot of frustration, because no normal users understand how to solve the problem. BUGFIX: Validate HTTPS Tracker Certificate by default (an0n666)

[arvidn]

I really don't understand why this is a bugfix?

This is a ticket reporting a bug. There's no fix for this anywhere (at least not linked to this ticket).

In fact, to quote myself:

I'm guessing there needs to be a special case for MacOS, just like for windows, where the system is queries for its root CA certs and explicitly added to the SSL context. Any help doing this would be appreciated. Surely someone else must have done this.

[aeular]

@arvidn I'm not sure this needs be so complex as special cases. Perhaps I'm vastly oversimplifying, but couldn't this be as simple as setting the SSL_CERT_ pair in the LSEnvironment section of the Info.plist for the mac app qBittorrent? Then the environment would be setup for openssl to find them on it's own and it all works.

[arvidn]

It could be even simpler by just adding the file /etc/ssl/cert.pem directly to the SSL context. I'm trying to understand where that file comes from though. Is that actually part of the OS and Apple's reponsibility?

[aeular]

I would suspect so, since it is the default used by openssl (aka LibreSSL) and curl on Big Sur that has had all of 2 apps total installed: qBittorrent and Transmission.

[arvidn]

transmission uses curl though.. but I suppose it wouldn't install things in /etc/ssl/

[aeular]

But I don't think transmission would change the default implementation of curl and openssl (LibreSSL) in terminal to use the /etc/ssl either

[arvidn]

this seems to work for me. Could someone else confirm? https://github.com/arvidn/libtorrent/pull/6094

[userdocs]

I think this is a linux user problem to. does this help?

https://golang.org/src/crypto/x509/root_linux.go

[arvidn]

are you experiencing this problem on linux? libtorrent asks OpenSSL to add the default path. So it should be a problem only if openssl doesn't know what the defaults are. I would expect distros to patch openssl to know about its default path.

[userdocs]

The stalling issue and the tracker failure appear to one and the same to me?

My trackers cannot be verified and so my trackers are "not working" so my torrent is stalled indefinitely.

It seems specific to the RC_1_2 branch and maybe the verification change in 4.3.4.1 is conflicting with RC_1_2 (1.2.13)?

I am on linux ubuntu/debina/alpine amd64

[arvidn]

@userdocs it's also possible the trackers that are failing for you don't have valid certificates (self signed or not rooted in some trusted CA). To tell those cases apart, could you test setting the environment variable(s) like above?

export SSL_CERT_FILE=/etc/ssl/cert.pem
export SSL_CERT_DIR=/etc/ssl/certs

(but to point to the path on your system)

[userdocs]

I'll try that and come back

[userdocs]

Ok so, using this service file

[Unit]
Description=qbittorrent
After=network-online.target
After=network-online.target nss-lookup.target

[Service]

Environment="SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"
Environment="SSL_CERT_DIR=/etc/ssl/certs"

Type=simple
ExecStart=%h/bin/qbittorrent-nox
Restart=always
RestartSec=2
TimeoutStopSec=5
SyslogIdentifier=qbittorrent

[Install]
WantedBy=default.target

It works

But if i don't use these env settings.

Environment="SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"
Environment="SSL_CERT_DIR=/etc/ssl/certs"

I get this instead

[userdocs]

I think I found my particular problem. Since I compile openssl statically I did this:

https://github.com/openssl/openssl/blob/master/Configure#L41

Setting it to --openssldir="/etc/ssl"

This seems to not require the env variables.