search

airdcpp-web / airdcpp-webclient

Communal peer-to-peer file sharing application for file servers/NAS devices
https://airdcpp-web.github.io
171 stars 31 forks source link

Consider system RAM when autodetecting max sim downloads #419

Closed aidan- closed 2 years ago

aidan- commented 2 years ago

Is your feature request related to a problem? Please describe. I have a box with a 1gbit connection but only 16GB ram. When configuring airdcpp, I set my Download speed (Mbit/s) to 1000 and left the Downloads limits on Detect automatically. I also left the Downloads -> Write buffer size (KiB) value at the default of ~524288 (IIRC).

It looks like this configuration results in the max downloads slots being set to around ~84. On a setup like this, it can cause OOM errors due to the number of ~500MB buffers kept in memory (84 * 500 = 42GB).

I believe this is the issue I hit, though it was quite difficult to diagnose as the system completely locked up (the flood of IO could also have been an issue on this box..). By manually configuring some lower values (half the slots and half the buffer size) I was able to stop airdcpp from crashing/freezing the system.

Describe the solution you'd like I took a quick look at the algorithm used to calculate download slots but got turned off by the comment:

    // Don't try to understand the formula used in here

:laughing: I think a reasonable solution would be to take into consideration the amount of RAM available on the system to ensure the calculated number of slots doesn't cause OOM.

Describe alternatives you've considered An alternative would be to perhaps put some more details/context (ie tooltips) in the WebUI config explaining how these values are used. That might be enough to inform users so they can make their own decisions on these values.

maksis commented 2 years ago

I also left the Downloads -> Write buffer size (KiB) value at the default of ~524288 (IIRC).

This can't be correct and I see no point in using such a huge buffer. The default write buffer size is 256 KB: https://github.com/airdcpp-web/airdcpp-webclient/blob/b064095c5480af2dec2e480e911a20849421258b/airdcpp-core/airdcpp/SettingsManager.cpp#L426

Do you have any idea that how it could have been set to 500MB?

aidan- commented 2 years ago

:facepalm: Yep, I see where/how this is being set now. I based my config off this ansible playbook and thought I stripped out all of the config, but I missed this line:

<BufferSize type="int">524288</BufferSize>

Thanks for the quick response - I'm going to remove this from DCPlusPlus.xml and let the default value be set. Apoligies for the time waste!

aidan- commented 2 years ago

Closing this - was 100% user error on my part. Changing the buffer size reduced the memory issue and also sped up downloads very very significantly.

Thanks again for your help, @maksis