TLS Certificate Error [including workaround]

[MiniXC]

Hello, thank you for providing this utility. When install anyflip-downloader, I encountered the following error:

Get "https://online.anyflip.com/.../mobile/javascript/config.js": tls: failed to verify certificate: x509: certificate signed by unknown authority

I managed to find a workaround by adding anyflips certificate to the trusted store as follows.

echo -n | openssl s_client -servername online.anyflip.com -connect online.anyflip.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/anyflip.cert
sudo mkdir -p /usr/local/share/ca-certificates/extra
sudo cp ~/anyflip.cert /usr/local/share/ca-certificates/extra/anyflip.crt
sudo update-ca-certificates

I'm not sure if there is a way to disable this certificate check in go, or if its even desireable, just leaving this here in case someone else comes across the same problem.

[Lofter1]

Hi, thanks for the notification. I could implement a flag that disables the check, but using the flag would be extremely risky, as one could not be sure whether they actually download from anyflip.

I'm not sure what exactly is causing this error in your case. Are you by any chance executing the tool inside a container (like a docker container) while also being behind a proxy that might change the certificate in a request? Like a corporate proxy?

If you could provide more information on what might cause this issue, we might find a better/more secure way to work around this. Otherwise, I will consider adding a "disable security checks" flag.

[MiniXC]

Afaik, I'm on a pretty standard Pop OS 20.04 setup, without a corporate proxy or anything like that.

[Lofter1]

my first thought would be because the release is a bit older and I'm not familiar how/when update for the cert store are triggered under Debian/Ubuntu based systems that your cert store might simply (have been) out of date or something along those lines. Assuming you use firefox, I think firefox does use their own cert store, and not the system store, by default, which would explain why your browser might not have had a problem. But this is only an assumption.

I'll leave the ticket open for now in case someone else might have this problem in the future. But assuming the above is true, I don't think a flag to ignore security features is necessary for now.

[mark-4dollar]

Same issue here with Fedora 37. Running from konsole, no vms/docker/proxy. Home network.

[Lofter1]

I'm not sure if this exists under fedora, but could you try to run sudo update-ca-certificates and see if the error persists? Fedora 37 released shortly before the current certificate of any flip was valid it seems.

And if the error persists, could you please post the results from curl -vvv https://anyflip.com (the top 20 or 21 lines should be enough I think)?

[mark-4dollar]

curl -vvv https://anyflip.com : https://pastebin.com/Vz9ASDSq

I exported their cert through brave (pem format) and added it to /etc/pki/ca-trust/source. ran sudo update-ca-trust, tried to run the app again, and received the same error:

Preparing to download 2023/07/11 16:45:08 Get "https://online.anyflip.com/jxpos/nvpk/mobile/javascript/config.js": tls: failed to verify certificate: x509: certificate signed by unknown authority

[Lofter1]

Yeah, sorry, but the received html doesn't help to identify what might be going wrong here at all. I need the actual verbose information, not the html.

[mark-4dollar]

It's exactly the output of what you wanted me to run.

Here is the output of: curl -vvv https://online.anyflip.com/jxpos/nvpk/mobile/javascript/config.js if that is what you are looking for.

https://pastebin.com/8qPurEpn

[mark-4dollar]

If I specify the cert in the curl parameters, I am able to successfully connect:

curl -vvv --cacert anyflip.pem https://online.anyflip.com/jxpos/nvpk/mobile/javascript/config.js

https://pastebin.com/5dkTqLpB

I added the pem (also converted it to x509 cer) to my local store. It shows as a user added certificate with common name *.anyflip.com - no luck with the app or curl (w/o the cacert param)

[Lofter1]

Sorry for the delay. I currently don't have much time to work on private projects. I will investigate this a bit further and see what I can do to avoid this issue or make a workaround easier.

To give a bit of insight: go uses the certificates in your machines certificate store. This might be out of date or missing certificates. curl for example also uses this store, that is why I asked to see a curl request, to absolutely make sure this is not a problem with the downloader or go. So it seems on some linux distributions do not trust the certificate from online.anyflip.com, whether that is because the store is outdated and needs to be updated or not intended to be trusted, I cannot say and/or might be different from distribution to distribution.

I will probably try to implement a flag similar to curls --cacert with a warning that this can come with a risk or similar. I'll try to do that this weekend, but I kindly ask to have a bit of patience regarding this issue, as I have a lot of stuff in my private life right now that also needs to be taken care of.

[Lofter1]

v0.1.5 now includes the -insecure flag. This should ignore the ca certificate checks. However, use this flag with caution, as it disables certificate checks for the entire program. While the described workaround by @MiniXC is not ideal from a security perspective, it is much preferred over using the -insecure flag.

I will keep this issue open, because I'd like to investigate this issue further and see whether there might be an even better workaround.

[MiniXC]

Would it be possible to run a test for this in GH actions, or is that overkill? Would at least make sure its not an issue in a standard linux vm.

[Lofter1]

GH Actions run in a docker container afaik, and that might lead to issues regardless as far as I have read online while researching this problem. Plus, I hope it would be a one time issue that do not need tests further down the line. If it turns out it might need tests in the future to not break stuff, I will implement some. In the meantime I will test in a normal VM on my machine.

I tried it on a Kali VM that I always have set up and could replicate it, however, that Kali was also severely outdated, and currently, because of a very bad internet connection, downloading a lot of stuff, like a new ISO, a Kernel Update or even a lot of package updates, takes hours. This will change in roughly a month and then I will investigate this further, but for now, your workaround as well as the new flag should suffice. At least I hope so.

[dakk]

I've found a better solution (for me):

echo -n | openssl s_client -servername online.anyflip.com -connect online.anyflip.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/anyflip.cert
SSL_CERT_FILE="anyflip.cert" ./go/bin/anyflip-downloader https://anyflip.com/....