Slow network downloads in Python3 3.7.7-15

[sladflob]

For new Package Requests, see the guidelines

Setup

Package Name: Python3 Package Version: 3.7.7-15

NAS Model: DS413 NAS Architecture: FREESCALE QorIQ P1022 DSM version: DSM 6.2.3-25426 Update 3

Expected behavior

Initially seen using sabnzbd. Downloads would generally max out my connection, which would be about 10 MB/s.

Actual behavior

After update to 3.7.7-15 downloads slowed down to 2.2 MB/s. Also verified by using a Python script directly.

Downgraded to 3.7.7-14 by manually installing python3.v14.f15047[qoriq].spk and maximum download speed was restored. To double check, upgraded to 3.7.7-15 again and speed dropped to 2.2 MB/s. Downgraded once again and back to full speed again.

Steps to reproduce

1. Upgrade Python3 to 3.7.7-15 2. Download using sabnzbd... download is very slow.

Alternatively use the script at https://github.com/sanderjo/python3-speedtest/blob/main/get_file_and_report_speed.py to see the slowdown with 3.7.7-15.

Issue was discussed in sabnzbd discussion forum when I initially thought it was a sabnzbd bug. See https://forums.sabnzbd.org/viewtopic.php?f=2&t=25136.

[BenjV]

This is because this version of python from the SynoCommunity for the first time support openssl correctly. So websites who support both http and https are now downloading via ssl encryption which cost performance

[sladflob]

I had slow downloads using a straight HTTP connection too.

[sladflob]

OK, I just verified that disabling SSL does indeed make the download faster. But that's an insane performance hit - from 10MB/s to 2.2 MB/s.

[BenjV]

The hit comes from both sites. The website will be slower and also the Nas needs more cpu to decode the sll encryption.

And using http will not stop all websites to use https, because they tend to try https first anyway and just fallback to http when the client does not responded to https.

[sladflob]

And yet with Python-3.7.7-14 and the download proceeding at full speed, sabnzbd tells me that it's using TLSv1.2 (AES128-SHA).

[BenjV]

That is impossible.

The whole reason of the python3 3.7.7-15 release was because before this release TLS was not supported by earlier release of python from the SynoCommunity. If you want to test it yourself you can use my python script for that. You can find it here.

https://github.com/BenjV/SYNO-packages/blob/master/GetInfo_Pyton3.py

EDIT: By the way I tested it on my Test Nas (DS116) with the same python script you posted a link to and got a download speed of 11.5Mbps

The file I used was: https://speed.hetzner.de/100MB.bin

Most likely your problem stems from the fact that you have a Nas with a antiek freescale processors that lacks support for hardware encryption.

[sladflob]

Ok so I ran your script and with Python3 3.7.7-14 installed I get the output at https://pastebin.com/5CFtwZ8P.

[BenjV]

Ok I made a mistake, TLS was not supported in versions prior to python3.7.7-14 TLS was introduced in python3.7.7-14 and update in python 3.7.7-15

The difference is the supported TLS version. As you can see in your output it with python3.7.7-14 the connection to the test sites are all TLS1 connections

tls-v1-0.badssl.com : ('ECDHE-RSA-AES256-SHA', 'TLSv1/SSLv3', 256) tls-v1-1.badssl.com : ('ECDHE-RSA-AES256-SHA', 'TLSv1/SSLv3', 256) tls-v1-2.badssl.com : ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1/SSLv3', 256) sha256.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128) sha384.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128) sha512.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128) www.github.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128) www.google.com : ('ECDHE-ECDSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128)

When I run my script with python3.7.7-15 I get this connections, so connecting with the max TLS version that teh site supports.:

tls-v1-0.badssl.com : ('ECDHE-RSA-AES256-SHA', 'TLSv1.0', 256) tls-v1-1.badssl.com : ('ECDHE-RSA-AES256-SHA', 'TLSv1.0', 256) tls-v1-2.badssl.com : ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256) sha256.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128) sha384.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128) sha512.badssl.com : ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128) www.github.com : ('TLS_AES_128_GCM_SHA256', 'TLSv1.3', 128) www.google.com : ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)

With python3.7.7-14 you can only use TLS1 and no higher version of TLS

My guess is that TLS 3 has more need of encryption support and your cpu does not have hardware encryption and has also very limited cpu power.

I am really puzzled why you have a drop in downloadspeed with python3.7.7-15 on a website that does not support https at all like http://speed.transip.nl as you said you have tested (did you really do this test?)

As I said I tested it on my DS116 and found no significant difference between downloaden from a https site or a http site with either pyton3.7.7-14 nor python3.7.7-15

[sladflob]

I definitely did do the test - several times. I will re-do it but can't do it for a little while. Perhaps it was something environmental as the tests between versions were a few hours apart.

Would you expect the speed drop to be as significant as what I'm seeing though? Also, the CPU meter on the Synology never goes above 30-40% when I'm using 3.7.7-15 which doesn't really indicate that the CPU is struggling.

[BenjV]

You have a dual core cpu so a single process can only use one core so 50% is the absolute max. You should do the test on this two sides to see difference between http and https.

http://speed.transip.nl/100mb.bin https://speed.hetzner.de/100MB.bin

[sladflob]

So I tried it twice for each site for each version of Python 3.

For 3.7.7-14: speed.transip.nl - 1.8 and 0.8 MB/s speed.hetzner.de - 6.5 and 6.8 MB/s

For 3.7.7-15: speed.transip.nl - 0.9 and 1.0 MB/s speed.hetzner.de - 2.1 and 2.2 MB/s

I've got a feeling that downloading from speed.transip.nl is generally very slow for me here in Australia, which may have been the source of my confusion. The https download seems to be hitting the same max that I'm getting in sabnzbd.

[BenjV]

Both sides are on the other side of the world so no supprise it is general slower then over here in Europe.

For http the speed are in the same order of magnitude for both python versions.. And the fact that https is slower stems most likely from the fact that for TLS 3 (only use which pyton3.7.7-15) the encryption is much heavier and your cpu is lacking hardware support for encryption.

[sladflob]

OK at least there's an explanation. Can I configure Open SSL at my end so that it doesn't try and use TLS 3?

[BenjV]

No that decisions is up tot the website. If it prefers TLS3 and your client support TLS3 then TLS3 it is. You could stay on python3.7.7-14 but in the near future websites are going to quit support voor lower version of TLS just as they are stopping support for http.

Numerous websites are already using TLS3 only. This is because lower TLS versions are not save anymore.

Maybe you should consider upgrading to a newer Nas in the near future?

[Safihre]

You could try to force a lower cipher in SABnzbd: https://sabnzbd.org/wiki/advanced/ssl-ciphers This one might be supported by your NAS.

[hgy59]

Python3 package is not maintained anymore.