search

iquercorb / OpenModMan

Open Mod Manager - Open source and generic Mod ("Modifications") manager.
GNU General Public License v3.0
95 stars 5 forks source link

Slow Download Rates #36

Closed schradj closed 2 years ago

schradj commented 2 years ago

Download rate is significantly slower than when accessing the file directly via URL through a web browser. Downloading a 795 MB file Via web browser: several tries all between 23-30 sec Via OMM: 2 min 49 sec Via OvGME: 10 min 36 sec Anything you can think of that may cause this? Same issue in OvGME, was hoping switching to OMM would remedy this. It is certainly an improvement, but if it could be as fast as downloading directly, that'd be awesome. Thanks

iquercorb commented 2 years ago

Absolutely no clue at first glance. OMM rely on libcurl for all the network part... As far as I know, libcurl is the most commonly used library, pretty well optimized. I will investigate, but I can't see how and why libcurl would be slower than browser.

schradj commented 2 years ago

Not a c++ developer, but maybe this is helpful

https://stackoverflow.com/questions/42537572/libcurl-download-speed-too-slow

iquercorb commented 2 years ago

Alright, there was indeed an issue. I even tested to download with the curl.exe and surprisingly it was even slower.

The problem came from the default size of download/write buffer that CURL use, which is pretty small (16kB) and adapted to regular network exchanges, but not very suitable for large download over a high-speed network. In such case, high transfer rate lead to very frequent I/O access for small writes, which become the bottleneck. The impact in negligible for download via an average internet connexion (rarely greater than 5MB/s) but become problematic with high-speed transfer (typically gigabit LAN).

Anyway, I increased the buffer size for file download, and now download rate is, it seem, the same as what browser perform. You can re-download and try the latest release (I did a new build and reuploaded installers but without increasing version number).