search

RustCrypto / password-hashes

Password hashing functions / KDFs
652 stars 80 forks source link

Argon2 - Freeing memory after use #478

Closed lfpraca closed 10 months ago

lfpraca commented 10 months ago

When using argon2, every time a password is verified, the memory used by the crate increases little by little until stabilizing at around 1.5GiB on my local machine (I am using the default configuration of 19MiB for argon2id) and remains allocated even after the verification is completed and the verifier is dropped. But most of the time it is more valuable to me if argon2 has less memory pre-allocated even if it takes a little more time to verify passwords. Is there a way to either automatically or manually free the memory after verifying a password? I couldn't find information regarding this in the documentation.

Thank you

tarcieri commented 10 months ago

Can you post a minimal self-contained repro?

The memory buffer is just a Vec which is freed on Drop at the end of this 2-line function: https://github.com/RustCrypto/password-hashes/blob/d25ea61/argon2/src/lib.rs#L230

Aside from that, there is nothing but stack allocations. The whole crate is heapless/no_std friendly. With SIMD disabled there is no unsafe code, even. I don't really see any place for a memory leak to occur in this crate.

Are you sure you're not hanging on to memory elsewhere, unrelated to this crate?

lfpraca commented 10 months ago

Thank you for the response. I don't think I am hanging on to memory. I created a minimal program to reproduce this behaviour and it is hanging on to less memory (as it is single-threaded) but still it is possible to see that before running the verifications the program is using almost no memory and after the verifications it is using ~200MiB, even after when the verifier should have been dropped.

main.rs.txt Cargo.toml.txt

tarcieri commented 10 months ago

I see it use 1GB of memory then drop down to 20MB of memory. Granted, this is on macOS.

Are you sure that isn't just memory which has been returned to the heap and hasn't been returned from the heap to the kernel?

lfpraca commented 10 months ago

I did a test turning the number down from 300 to 3 verifications per run and running them 80 at a time, I waited for the 80 that I ran to complete before starting a new batch of 80. I could eventually make my computer (running linux) run out of memory and crash, which makes me think this is not what is happening, but I could be wrong

lfpraca commented 10 months ago

I tried changing the program to allocate memory with something else, I used a large vector. With it the memory went up to ~180MiB then down at the time "Freed" is printed. In the end the program was using the same amount of memory as when it started

main.rs.txt

tarcieri commented 10 months ago

There is literally nowhere else in argon2 crate that touches dynamic memory than other than this: https://github.com/RustCrypto/password-hashes/blob/d25ea61/argon2/src/lib.rs#L230

I simply don't see any room for this crate to be at fault. It's just allocating a vec! and immediately dropping it when it's done.

Perhaps report this upstream to https://github.com/rust-lang/rust ? Otherwise I don't see anywhere a bug could possibly exist in this crate.

tarcieri commented 10 months ago

I tried running this on Debian 12 in a Docker container with a Linux 5.15 kernel and glibc 2.36.

RSS as reported by ps seemed fairly constant at around ~8000, going up to ~9000 then back down to ~7000 and repeating. This is with your original example.

I'm unable to reproduce this at all. It seems like something with your environment?

tarcieri commented 10 months ago

Closing as unreproducible (and even if it were, I don't see anything actionable here on this crate's part)