(Suggestion) Make SHA-512 be the one that's tested first when autodetection is selection.

[nikkopt]

Greetings. So, when i mount a volume and i manually select the PKCS-5 (SHA512 because it's the default when creating volumes), the volume mount's in like 1.5 seconds. When "Autodetect" is selected, it takes like 15 seconds to mount which makes me think that the default one for creating volumes isn't the first one to be tested on the list. Is there a good reason for this 15 seconds wait time? Can't SHA512 be the first to be tested since it's de default one? Thanks.

[alt3r-3go]

There's a feature for that, under Default Mount Parameters (see here: https://www.veracrypt.fr/en/Default%20Mount%20Parameters.html) - you can set the default PRF to try first right there. Does that help?

[nikkopt]

It does. I feel dumb missing that. Thanks

[cypherbits]

Setting the Default Mount Parameters and making SHA-512 to be the first tested when Autodetection is selected is a different thing. I think this should be reopened. When you first create a volume, the default selected option is SHA-512, so if you select Autodetection here, the first checked should be in the same order.

[nikkopt]

I agree, although i think setting it in the options is a good workaround. I'll reopen it anyway because it does make sense.

[alt3r-3go]

Looks like this is going to be fixed with https://github.com/veracrypt/VeraCrypt/commit/fdf7888ab3675a267e06e2f3acceeedfa5b74f62

[idrassi]

I confirm that a change was implemented to make PRF auto-detection nearly fast as manual selection of the PRF. The newer commit https://github.com/veracrypt/VeraCrypt/commit/4b98ff0e9810a218f802d08cfd546c2fd67757dc brings better code than previous commit mentioned by @alt3r-3go.

I have uploaded the first beta of 1.24-Update9 containing this enhancement to Sourceforge Nightly Build folder (look for VeraCrypt Setup 1.24-Update9-Beta-21-08-07.exe or VeraCrypt Portable 1.24-Update9-Beta-21-08-07.exe). Binaries are signed with official keys as usual.

Tests and feedback are welcomed.

[nikkopt]

I only updated veracrypt today since i've been busy. I'm on 1.25.7 but the speed didn't seem to change for me for some containers. I have a 65GB and a 250GB container and it takes 13-15 seconds for them to open on auto. I have another one with 1.4GB that opens as fast as auto when set to manual. Should their sizes influence the speed?

[nikkopt]

Ok i just made some tests. 2 containers 1GB each, one formatted as FAT and the other as NTFS. The FAT one opens in 1-2 seconds, the NTFS one needs almost 15 seconds.

[idrassi]

Thank you for the update. I assume both FAT and NTFS volumes use the same encryption parameters. The difference between FAT volumes and NTFS volumes is strange because VeraCrypt doesn't have any knowledge about the filesystem: it just unlocks the volume so that we handle Windows low level I/O on the fly.

To me, it looks like Windows Mount Manager needs 13 seconds to make the NTFS filesystem of the volume available to the system. We don't have control over this part and on my various test systems I don't have such difference.

So certainly something on your PC is taking over NTFS volumes and performs some kind of operations before they are made visible to the system. And probably the culprit has some issue with NTFS volumes if they are located in an VeraCrypt volume.

Until we know who is responsible for this delay, we can't really help. You can try to check Windows Event Viewer to see if there is an event logged during or after the 13 seconds period. This could give us a hint about the culprit.

[nikkopt]

Yes both are using the same encryption parameters. The NTFS containers only take 13-15 seconds if i choose auto, they mount in 1-2 seconds if the correct option is chosen. Windows event viewer doesn't help much, it just says Volume Z: (\Device\VeraCryptVolumeZ) está em bom estado de funcionamento. Não é necessária nenhuma ação. which means that volume is healthy and no action is needed. I don't know if this helps but when veracrypt is mounting those volumes, the kernel spawns 3 veracrypt processes (each using one thread at 100%) as you can see here: No disk access.

[idrassi]

Thank you for the detailed analysis.

The 3 threads that use 100% CPU each are part of the threads used by VeraCrypt to try all possible hash algorithms in parallel (SHA512, SHA256, Whirpool, RIPEMD160 and Streebog).

I guess you took this screenshot after 2 seconds of the beginning of mount. In this case, the 3 threads correspond to the slowest hashes (Whirpool, RIPEMD160 and Streebog) and normally SHA512 and SHA256 have already finished which means that mounting should have returned.

How many virtual processors does your CPU have? Can you please post screenshot of VeraCrypt benchmark of PKCS5-PRF (menu Tools->Benchmark then choose PKCS5-PRF)? Did you limit the number of threads to use in VeraCrypt settings?

I hope this will give me some idea on why NTFS is affected by this.

Last query: do you have the same delay if you mount the NTFS volumes as removal media (Mount Options -> Mount volume as removable medium)?

Thanks.

[nikkopt]

used by VeraCrypt to try all possible hash algorithms in parallel

You're right, it starts with 5. It just seems to mount when all of them end.

https://user-images.githubusercontent.com/2060960/153775434-62dc1dad-b583-4ba9-8114-86c43bc98212.mp4

How many virtual processors does your CPU have?

16. Dual threaded 8 cores.

Can you please post screenshot of VeraCrypt benchmark of PKCS5-PRF (menu Tools->Benchmark then choose PKCS5-PRF)?

Did you limit the number of threads to use in VeraCrypt settings?

Just checked afinnity, it's using all of them. For fun i tried with just 4 and it was the same. I may be wrong but i don't think it matters. When it's mounting, the veracrypt.exe process isn't using any cpu resources, in the windows task manager it's a process called "System". Using the app "TaskExplorer" (the one i screenshot in my last post), i can actually see those veracrypt.sys child processes that were spawned by the kernel.

do you have the same delay if you mount the NTFS volumes as removal media (Mount Options -> Mount volume as removable medium)?

Yep, same thing

[idrassi]

Thank you very much for the video and the information. The thread pool mechanism works as expected but somehow the notification that SHA512 has finished is not received by the mounting thread. I'm still scratching my head on why this happens and why only with NTFS volumes. I will review the source code in case I can find something. I will keep you informed.

[nikkopt]

Thanks a bunch. By the way, i only tested with FAT and NTFS, not the other filesystems. I'll try them tomorrow since i'm not at the computer at the moment.

[nikkopt]

It seems it only affects NTFS (I can't try ReFS, it gives me an error saying windows couldn't format it). No filesystem, FAT, FAT32 and exFAT mount instantaneously. If i mount the NTFS container and then format it to another filesystem through the explorer, it will begin mounting instantaneously. If i format it again to NTFS, it will take again 13-15 seconds to mount. This makes one think that it's a windows problem. However, wouldn't it also behave like that if i chose SHA512 manually? It doesn't. It mounts instantaneously if the hash is chosen. Is there anything else i can try to help troubleshoot the problem?

PS: For the containers that mount instantaneously, the threads trying the other hashes keep going and only stop when they fail, not when the right one is found and the container is mounter. This is by design?

[nikkopt]

I did some more tests.

On a windows 7 virtual machine with 4 cores: Manual:~1 sec Auto:~10 secs (yep, it's a bit faster on a VM with less cores than on the host machine)

On an old dual core celeron netbook running a very old version of win10: Manual:~4 secs Auto:~18secs

These machines have nothing in common other than being mine. By the way, i can greatly decrease the time it takes to mount on my main machine from 15secs to around 6 by changing the power profile (which locks the cpu at a max 2.2GHz) from power saving to balanced or performance. But by doing that, the 1 second it takes to mount a volume by manually choosing SHA512 drops literally to instantaneously. What i'm trying to say is that the difference between auto and manual in percentage stays the same.

[idrassi]

Thank you for all these tests, I appreciate your help. The Windows 7 VM test is just incredible! But this can be caused by Windows 7 being more optimal than Windows 10 in such use case.

As you noted, it is strange that the issue with NTFS disappears when using manual PRF selection. It seems as if Windows is doing something special for NTFS volume and it is somehow blocking VeraCrypt main thread until all its sub threads finish their processing.

I will try to reproduce the issue on my various machines. If I can, then I will be able to generate driver traces and from there look for the culprit.

Concerning your question about the other threads continuing their processing even if the right hash was found, this is indeed by design since there is no way to stop these threads in the middle of the job and so we just leave them until they fail.

[eugenesvk]

I've just checked on an old Macbook OSX 10.15 Veracrypt v1.25.9 (with macfuse 4.4.1) with a small 50Mb container with both hidden and outer volumes formatted as FAT, and I observe exactly the same issue - the hidden volume autodetection is ~20 seconds while selecting SHA-512 manually takes ~2 seconds The outer volume doesn't have this difference, both autodetection and manual detection take ~2sec (both volumes are encrypted with a single-digit password)

Sorry don't know how to do any proper tests on a Mac, so just a couple observations based off activity monitor I see two procs here, one with ~10 threads, another one with 1 While trying to mount:

• the ~10-thread proc increases the number by a few threads in both instances,
• the 1-thread proc behaves differently, but the only difference is in how % of CPU and time it takes (autodetection is much longer)