Slow Network speed for large file transfers

[NaXal]

Hello,

I have a home server running two VMs, Jellyfin and another NextCloud on bridge networking mode. The Host OS is Windows and hypervisor is VMWare.

I wanted to expose both to Internet so deployed a 3rd VM running Ubuntu 20 and then followed the installation instruction to install this software (Nginx Proxy Manager).

Following are some configuration details

Hardware -> Intel Core i7 4790 / 32GB DDR3 / SATA SSD / Gigabit LAN Host OS -> Windows 11 Hypervisor -> VMWare 16

Router -> 192.168.1.1 (having the public IP) Ports -> 80/443 is forwarded to Nginx VM

Host OS -> 192.168.1.25 VM-Jellyfin -> 192.168.1.60 VM-NextCloud -> 192.168.1.30 VM-Nginx -> 192.168.1.45

Nginx Host added as subdomain-jellyfin.mydomain.com -> goes to jellyfin and similar 2nd subdomain for nextcloud too.

Setup works. Both these VMs (Jellyfin / NextCloud) are accessible from outside internet via their respective subdomains.

However large file transfers are really really slow when going via this Reverse proxy VM. For example, both NextCloud or Jellyfin VMs are unable to sustain more than 15-20 megabits. The transfer (upload or download) starts fine with my connections max speed, but gradually within few seconds, it starts to slow down and settle at around 15-16 megabits / sec.

Just to compare, if I cut the middle proxy VM by directly exposing the Jellyfin or Nextcloud VMs to the internet via router port forwarding, both of them are able to run at full speed of my connection (250 megabits / sec). Be it upload or download.

Since this Proxy Manager is web UI based, so chance of me messing up anything is slim. I have not enabled SSL in any of my VMs. I have not added any extra or advance config argument to the proxy manager UI.

So what should I do now to troubleshoot this performance issue?

Thanks in advance.

[SolarisEclipse]

Try lighttpd and see if it works better instead of nginx, then try apache. It could be that what's happening is you don't have the computing power to push the full uncompressed file through the internet. You should be compressing files to make it easier to transfer the reason why Jellyfin is fine is because it compresses the video. Also you shouldn't compare your amateur solutions to big corp solutions, they have CDNs and redundant instances via docker. (You should really consider switching to docker, pod man, containerd, instead of having everything in their own VM it's way more efficient)

[SolarisEclipse]

I notice you haven't responded or anything but HAProxy is another easy to set up load balancer/proxy solution.

[NaXal]

Hello,

Pardon me for the delayed response. I was & still spending time with the setup to tune it up for best possible performance.

Here is the update,

Based on your first reply, I started digging.

It could be that what's happening is you don't have the computing power to push the full uncompressed file through the internet.

I doubt that. At least from the Host Windows resource usage reporting. CPU isn't getting bottlenecked. I have a dedicated video card for Jellyfin media transcoding so playback over internet is dead smooth without any load in CPU. The issue was never that, rather it was when I am trying to download a large file, speed was dead slow when going via NPM

Also you shouldn't compare your amateur solutions to big corp solutions, they have CDNs and redundant instances via docker.

I am not at all comparing myself with any large corp with high end hardware. But things work as they are supposed without the proxy in between and things are slowing down (specially the large file transfers) when putting the proxy in between. That is what I am trying to troubleshoot

(You should really consider switching to docker, pod man, containerd, instead of having everything in their own VM it's way more efficient)

Everyone has their own need based on the available resource. So based on my setup, VM works the best for it's untouched portability feature. It's really easy for me to move or backup the entire VM without much of a technical challenge.

Afterall, for non technical users like me, this GUI based Proxy Manager felt the easiest and best option to go for over HAProxy or any other solution.

Anyway, coming to my issue, I have made some progress,

I did few changes based on your views. I have moved the Jellyfin out from VM and into the Host OS directly. Can't understand why it effected things, but speeds are much better and usable now. Still not the fastest or stable but I am able to archive 80/90 Mbits with the proxy in between.

However that landed me into another issue. Large file download / uploads were erroring out at 1GB mark. Searched for the same over the internet & that pointed me towards proxy_max_temp_file_size. As per nginx documents, I guess the default size is 1GB, so added that extra argument in Proxy Manager GUI panel advance settings and set the value to 16GB (max what I or my users would need). Further, I increased the NPM VM disc to allow such large file writing. That solved the issue of erroring out.

I am still searching & reading about the speed fluctuations, what further can be done or where to look for fixing it.

Thanks.

[lue30499]

Hi,

I had to put this: proxy_request_buffering off; client_max_body_size 0;

in my advanced part of the proxy host to get nextcloud to send files at normal uninhibited speed.

See if that works.

Regards, Lue.

[kabaga]

Hi,

I had to put this: proxy_request_buffering off; client_max_body_size 0;

in my advanced part of the proxy host to get nextcloud to send files at normal uninhibited speed.

See if that works.

Regards, Lue.

I am having the same issue where I could not upload bigger than 100MB. I added those lines under the Advanced tab, and restarted NPM, but nothing has changed.

[lue30499]

@kabaga Have you checked your php memory limit in nextcloud? This thread is more specific to it being fine directly to nextcloud but not behind a proxy.

[kabaga]

@lue30499 Yes, I added these two lines to my docker-compose file. I also checked it also shows in the web UI those values

      - PHP_MEMORY_LIMIT=8G
      - PHP_UPLOAD_LIMIT=10G

[NaXal]

Hello,

It's bit sketchy but as suggested here, following settings under advance configuration are somewhat working for me,

proxy_request_buffering off;
proxy_max_temp_file_size 10240;
client_max_body_size 0;

Thanks.

Update (30-06-2022)

Something broke with my Ubuntu 20 + Nginx Proxy Manager Docker as no matter which ever configuration (including the above mentioned) I tried, my large upload / download speeds were down crawling.

Reinstalled the setup with Ubuntu 22 + Docker CE and Docker Compose 2.6.1 and then deployed Nginx Proxy Manager yml image. -> With this new setup, large file upload and download speeds are much better but still not reaching my connections top speed. Tried fiddling with few custom configuration parameters and this one single line of configuration seems to be doing the trick,

proxy_buffering off;

Thanks.

[github-actions[bot]]

Issue is now considered stale. If you want to keep it open, please comment :+1:

[DaHokeyPokey]

Hi, I think the issue is because you have your NPM as a bridge connection. I would recommend setup up a macvlan network. This would register the NPM on your local network with its own mac address. This should allow you to have your max upload/download speed for Nextcloud.

[anderconsu]

Hi, I think the issue is because you have your NPM as a bridge connection. I would recommend setup up a macvlan network. This would register the NPM on your local network with its own mac address. This should allow you to have your max upload/download speed for Nextcloud.

Thanks, I'll try that when I can