ISP throttles my storj connection (uplink/rclone). Any workaround?

[bitcoinmeetups]

Hello,

I'm BM. A very nice and polite guy.

It seems like my ISP throttles my storj connection (uplink or rclone). Any workaround?

If I try to upload a large file for testing purposes I will go from 2MB/s to 1.5 MB/s and all the way to zero. Then I get cut off from the Internet temporarily. Perhaps this due to the large amount of TCP connections.

Anyone knows how to fix this?

[thepaul]

Maybe!

That sounds like a bufferbloat problem to me. What OS are you running with uplink and rclone?

[Erikvv]

More likely your home router is crashing.

[bitcoinmeetups]

Termux or Raspberry Pi OS. Same issue.

[bitcoinmeetups]

More likely your home router is crashing.

Teragul. I have no router. Mobile phone internet connection.

[thepaul]

The router isn't crashing- it just has a full egress buffer (because you have a lot of outbound traffic, and you can send bytes to the router more quickly than it can send them over the Internet). And because buffers are so big in routers nowadays, it can take many minutes for a packet to make it through the queue. That wouldn't be so bad, except ACK and SYN packets and DNS queries all have to wait their turn in the buffer as well. This makes it look like you are cut off from the internet entirely. DNS lookups and new browser connections time out, and existing connections trying to download get stalled because their ACK packets are stuck waiting in the buffer.

This phenomenon is extremely common on consumer grade routers and networks, where the huge majority of traffic is downstream. It is called bufferbloat.

I don't know if there's anything to be done with respect to Termux (it looks like that's just a terminal emulator on Android), but on the Raspberry Pi you can enable the LEDBAT congestion controller, and that should solve the problem.

cat /proc/sys/net/ipv4/tcp_available_congestion_control

should show you the available congestion controllers. If ledbat is in that list, you can enable it with

sudo sysctl -w net.ipv4.tcp_congestion_control=ledbat

[thepaul]

I have no router. Mobile phone internet connection.

In that case, the culprit could be your phone itself, or whatever switch or router is your next hop. I've observed bufferbloat on a cell network before, so I know it can happen, but I want to believe it doesn't happen as often as it does with home routers.

Try LEDBAT and see if the problem goes away.

[bitcoinmeetups]

It says reno cubic

Sent from ProtonMail mobile

-------- Original Message -------- On Aug 11, 2022, 04:52, paul cannon wrote:

The router isn't crashing- it just has a full egress buffer (because you have a lot of outbound traffic, and you can send bytes to the router more quickly than it can send them over the Internet). And because buffers are so big in routers nowadays, it can take many minutes for a packet to make it through the queue. That wouldn't be so bad, except ACK and SYN packets and DNS queries all have to wait their turn in the buffer as well. This makes it look like you are cut off from the internet entirely. DNS lookups and new browser connections time out, and existing connections trying to download get stalled because their ACK packets are stuck waiting in the buffer.

This phenomenon is extremely common on consumer grade routers and networks, where the huge majority of traffic is downstream. It is called bufferbloat.

I don't know if there's anything to be done with respect to Termux (it looks like that's just a terminal emulator on Android), but on the Raspberry Pi you can enable the LEDBAT congestion controller, and that should solve the problem.

cat /proc/sys/net/ipv4/tcp_available_congestion_control

should show you the available congestion controllers. If ledbat is in that list, you can enable it with

sudo sysctl -w net.ipv4.tcp_congestion_control=ledbat

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[thepaul]

sudo modprobe tcp_ledbat might load the module so it's available.

If not, try sudo modprobe tcp_lp and then use lp in place of ledbat. That's more widely available and might do the job.

[bitcoinmeetups]

error messages

4> sudo modprobe tcp_ledbat modprobe: FATAL: Module tcp_ledbat not found in directory /lib/modules/5.15.32-v8+ 4> sudo modprobe tcp_lp modprobe: FATAL: Module tcp_lp not found in directory /lib/modules/5.15.32-v8+

[thepaul]

I guess you don't have any extra congestion controllers. You may need to build and install https://github.com/silviov/TCP-LEDBAT.

[bitcoinmeetups]

If the user needs to build and install congestion controllers to use the product then it might be time to fix the issue.

Please try to make storj work on Termux and Raspberry Pi. I have tried to build LEDBAT already but I got error messages.

But I feel like it's not a good idea to lock out 1 billion plus potential users. As far as I understand storj does not work on ARM, mobile or RPi at the moment due to this issue.

On a separate note, I think storj seems like an interesting package and I really want to test it. I was enthusiastic to see that rclone now supports it.

Can you please make a software update on storj itself that solves this issue?

[thepaul]

I'm afraid there is only so much we can do. The root of the problem is the large buffer sizes and lack of Smart Queue Management on consumer-grade routers. These problems are fairly common, so we try to provide workarounds where we can, but we can't reconfigure your network or override the way the network stack works in your kernel.

There are quite a few people running Storj nodes on Raspberry Pi hardware (I'm one of them). Storj (both the storage node software and the "uplink" client library) work fine on ARM (ARMv5 or later, as far as I know) and ARM64.

We do have some projects going internally that could let us bypass the tcp stack and implement congestion control in our own software, for example by using the Micro Transport Protocol (µTP) or by porting LEDBAT into an appropriate implementation of QUIC. But it's unclear how long those will take or how successful they'll be.

For the time being, maybe we can help with your LEDBAT setup. What errors are you getting?

[bitcoinmeetups]

Ok, I will make a third or fourth attempt to make LEDBAT work with Storj. I will update with the error messages as soon as possible, cannot remember what they were at the moment.

[bitcoinmeetups]

Well, this could be trivial but I have googled it a few times and not found the solution yet.

build-essentials is already installed

cd /TCP-LEDBAT/src

make

Outputs:

make -C /lib/modules/5.15.32-v8+/build M=/hom

e/pi/TCP-LEDBAT/src modules

make[1]: *** /lib/modules/5.15.32-v8+/build:

No such file or directory. Stop.

make: *** [Makefile:6: default] Error

-------- Original Message -------- On Sep 2, 2022, 04:38, paul cannon wrote:

I'm afraid there is only so much we can do. The root of the problem is the large buffer sizes and lack of Smart Queue Management on consumer-grade routers. These problems are fairly common, so we try to provide workarounds where we can, but we can't reconfigure your network or override the way the network stack works in your kernel.

There are quite a few people running Storj nodes on Raspberry Pi hardware (I'm one of them). Storj (both the storage node software and the "uplink" client library) work fine on ARM (ARMv5 or later, as far as I know) and ARM64.

We do have some projects going internally that could let us bypass the tcp stack and implement congestion control in our own software, for example by using the Micro Transport Protocol (µTP) or by porting LEDBAT into an appropriate implementation of QUIC. But it's unclear how long those will take or how successful they'll be.

For the time being, maybe we can help with your LEDBAT setup. What errors are you getting?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[thepaul]

Oh ok! I think you just need to apt install raspberrypi-kernel-headers.

[bitcoinmeetups]

I tried installing raspberrypi-kernel-headers but I still get "build no such file or directory" when i try to make in the src folder.

[thepaul]

That suggests that your running kernel does not match the kernel version provided by raspberrypi-kernel-headers. You probably just need to reboot to get the new kernel. If that isn't an option, you'll need to find an older copy of the raspberrypi-kernel-headers deb that provides headers matching your running kernel.