Add ability to disable parallel file transfers

[nvonbulow]

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

When writing multiple files to an HDD at a time, files will transfer extremely slowly. However, this performance can be improved by eliminating random writes by reading just one file at a time serially. This should reduce (but not eliminate) random IO operations.

Describe the solution you'd like A clear and concise description of what you want to happen.

I propose a setting that will disable async file transfers completely without a size limit. However, this feature does not need to be removed completely, since it still has a benefit when both the source and destination are SSDs. When moving small files (<1MB), I have gotten as few as 2 files per second, and the disk response time is ~500ms.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

An alternative that requires less user intervention is detecting which type of drive each library is on. If both are SSDs, then enable async file transfers, otherwise disable them.

Additional context Add any other context or screenshots about the feature request here.

This will not improve performance for SSD drives, since they are suitable for random IO operations.

[nvonbulow]

Example Game: The Jackbox Party Pack has a ton of small files (the average file size is ~100KB, but many are <1KB).

When transferring this from an SSD to an HDD using SLM, I was getting ~2 files/s at the worst, but when I did the same transfer using Windows Explorer, I was getting ~50 files/s at the worst.

[RevoLand]

Hello,

First of all thank you for your suggestion. I've implemented it in a way you can enable/disable per-task like rest of the other task options. (However we need to make adjustments to the task manager ui before it goes public as we are getting lesser space with every customization option)

You can download a pre-compiled version of unreleased changes from here and test.

The issue i had for a long time is being unable to take proper benchmark results as interrupted by caching and some other unevidable stuff. I've been looking for a way to properly bench the transfer speeds for a long time, even closed every caching related stuff i could find however, i am still unable to bench the transfer speeds properly/repeatable.

When you can't get repetitive results, you can't even properly measure to see how does ui updating effects the transfer speeds. However, from what i can tell you and bech'ed personally with Left 4 Dead 2 (lots of small files too), the current method is the best one i could've came with.

I will be still trying to find a way for a proper measurement to improve the file transfers speeds, thank you again for the suggestion, keep it flowing! :)

[nvonbulow]

Hi there, thanks for the quick response.

It seems that I misunderstood what an async file transfer was for some reason. All it means is that it's not done on the main thread. It turns out that the major factor for how well small files transferred was the antivirus software. I didn't see a significant difference between async toggled on or off. Once I turned off real-time protection for Windows Defender, the files transferred a lot faster (8MB/s vs 500KB/s). I think that you should insert a tip to turn off antivirus software during transfers to speed up transfers, especially for games with small files.

[RevoLand]

Hello and thanks to you for using SLM :)

All good, i thought you meant transferring files in parallel or synchronously at first place and made the implementstion in that way, with wrong naming tho :)

On the other hand, even tho released versions are never obfuscated and all sources publicly avaiilable; wouldn't suggesting for disabling anti-virus look suspicious, for some at least?

[nvonbulow]

Yeah, I suppose it could look suspicious, but if you put an explanation of why in the wiki and link to it, then that would satisfy many people.

[RevoLand]

Allright, thanks!