Perf issue on external usb-c ssd

[pzwahlen]

Hi,

I'm trying to understand the root cause of a performance issue when writing to an external USB-C drive through SecureFS+WinFSP . This happens on Windows 11 with securefs 0.14.2 and 0.14.3, as well as WinFSP 2022 and 2023.

I have a local/internal NVMe M.2 SSD as well as an external USB-C SSD (Samsung T5 EVO)

My reference write speed is when writing a 1G file with random content. On both my local SSD and USB-C SSD, this happens at ~300MB/s

If I create a local SecureFS volume on my internal SSD, I can push this file through SecureFS+WinFSP at the same speed, ~300MB/s. Encryption has basically no impact.

However, if I create a volume on my external USB-C SSD, then the throughput drops to ~3MB/s, a 100x decrease.

To me the culprit seems to be WinFSP when dealing with external USB drives (as I would say SecureFS is not aware of it), however I'm curious to have some feedback here (while waiting to be accepted into the WinFSP Google group).

Thanks, and congrats for SecureFS. Patrick.

[netheril96]

Can you give some specifics?

• Your .securefs.json file with keys stripped
• The command you run to mount the volume
• How you test the random access speed

[pzwahlen]

Sure, and thanks for the quick reply.

I'm using SiriKali with all defaults. The resulting json config is

{
    "argon2_m_cost" : 262144,
    "argon2_p" : 4,
    "block_size" : 4096,
    "encrypted_key" : 
    {
        "IV" : "9b897e06406816992e5fdc0400910f5ebbf6d7cf7ded416c497b69e807134999",
        "MAC" : "26735dbc34e273cbf08309764cd90135",
        "key" : "<stripped>"
    },
    "iterations" : 9,
    "iv_size" : 12,
    "pbkdf" : "argon2id",
    "salt" : "57bcd7559d76a74c3a26164870b5f79e2c1644e94c9274566bb181c4e4a6f578",
    "version" : 4
}

SiriKali is mounting the volume with:

securefs.exe" mount --fsname Securefs@D:/test2 --fssubtype Securefs D:/test2 Y: -o rw,volname=test2

D: is my external USB-C drive. On this volume that I just re-created, I have 3MB/s write speed and about 10MB/s read speed.

So far I didn't test random access speed, only throughput.

[netheril96]

1. How do you test for read/write throughput?
2. Have you enabled disk caching for the external drive?

I have an external HDD. On Windows I can achieve 10+MB/s write speed with securefs, so I find your experience a bit difficult to comprehend.

[pzwahlen]

Yeah, my issue is a bit hard to comprehend, but I think your questions made me move forward. Thanks for that.

1. I am testing throughput by simply drag'n dropping a 1G file (with random content) on top of my external SSD or on top of a SecureFS volume sitting on that same external SSD. I should maybe add that the drive is formatted as exFAT.
2. Config is "all default" and drive is configured for "quick removal" which I think disables some caching.

Based on the above, I disabled "quick removal" for this drive and made sure write caching was enabled. Speed is day and night and that was probably the root cause of my issue.

However, I have to say that dokany+cppcryptfs was perfectly happy with this config and able to provide ~200MB/s write throughput.

I'm pretty sure this has nothing to do with SecureFS and is more a WinFSP behaviour.

[netheril96]

Can you mount with option --trace and see if there are many fsync calls? Both when "quick removal" is enabled and disabled.

I'm thinking that the constant flushing may be the cause.

[pzwahlen]

This link contains 2 traces: https://cloud.lesz.ch/s/2z3qMJrmeZAG3Zz

Both traces show the mount, copy of 1G.bin and unmount.

trace_ok is when it's working fine, ie:

• Quick removal: disabled
• Write caching: enabled

The copy took place at ~300MB/s, trace file containe 5.7k lines

trace_nok is when it's not working, ie:

• Quick removal: enabled
• Write caching is greyed out and cannot be configured

This copy took place at ~3MB/s, trace file contains 22k lines.

I searched for fsync and couldn't find it, but maybe you will find something else.

[netheril96]

One potential cause is that securefs is issuing Write on each block separately, rather than consecutively. I may fix it in future releases.

For now, make sure you turn on caching. It's good for the performance of other software too.

[pzwahlen]

Thanks for the analysis.

Going to close this issue!

[netheril96]

Can you try if this experimental new release is any faster? securefs.zip

[pzwahlen]

Can you try if this experimental new release is any faster? securefs.zip

This is day and night. This build is now more or less working as I expected (without changing caching/eject policies).

Is it safe to use ? For the moment I have reverted to the official build.

Thanks for looking into this!

[netheril96]

I need more testing before releasing it. Please use official build for important data for now.

On Mon, Mar 18, 2024 at 01:26 pzwahlen @.***> wrote:

Can you try if this experimental new release is any faster? securefs.zip https://github.com/netheril96/securefs/files/14624256/securefs.zip

This is day and night. This build is now more or less working as I expected (without changing caching/eject policies).

Is it safe to use ? For the moment I have reverted to the official build.

Thanks for looking into this!

— Reply to this email directly, view it on GitHub https://github.com/netheril96/securefs/issues/170#issuecomment-2002546582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGMJZ2QZAV4JCY6J7GODCLYYXG45AVCNFSM6AAAAABCYITCECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGU2DMNJYGI . You are receiving this because you modified the open/close state.Message ID: @.***>

[netheril96]

Fixed in release v1.0.0.