search

alexedwards / argon2id

Argon2id password hashing and verification for Go
MIT License
452 stars 44 forks source link

huge memory allocation #3

Closed bruno-nascimento closed 4 years ago

bruno-nascimento commented 4 years ago

// curl http://localhost:7777/debug/pprof/heap > heap.out // [bruno@localhost wrk2]$ go tool pprof heap.out // File: main // Type: inuse_space // Time: Oct 31, 2019 at 10:46am (-03) // Entering interactive mode (type "help" for commands, "o" for options) // (pprof) top // Showing nodes accounting for 6.94GB, 100% of 6.94GB total // Dropped 7 nodes (cum <= 0.03GB) // flat flat% sum% cum cum% // 6.94GB 100% 100% 6.94GB 100% golang.org/x/crypto/argon2.initBlocks // 0 0% 100% 6.94GB 100% github.com/alexedwards/argon2id.ComparePasswordAndHash // 0 0% 100% 6.94GB 100% github.com/valyala/fasthttp.(Server).serveConn // 0 0% 100% 6.94GB 100% github.com/valyala/fasthttp.(workerPool).getCh.func1 // 0 0% 100% 6.94GB 100% github.com/valyala/fasthttp.(*workerPool).workerFunc // 0 0% 100% 6.94GB 100% gitlab.com/???/???/endpoints.Login // 0 0% 100% 6.94GB 100% gitlab.com/???/???/infra/server.handler // 0 0% 100% 6.94GB 100% gitlab.com/???/???/model/users.Authenticate // 0 0% 100% 6.94GB 100% golang.org/x/crypto/argon2.IDKey // 0 0% 100% 6.94GB 100% golang.org/x/crypto/argon2.deriveKey

alexedwards commented 4 years ago

In what scenario did you see this? During a load test, or under normal use in production?

The Argon2 algorithm uses a lot of memory by design --- the more memory required the higher the cost of calculating/verifying the hash. The default setting for this library is to use ~65MB for each calculation, so if you perform 100 calculations in quick succession you will end up with about 6.5GB of memory being allocated (and eventually it will get garbage collected).

You can reduce the memory parameter to reduce the cost, if this is too much.

Or is there something that makes you think there is a problem with this package or the golang.org/x/crypto/argon2 implementation, which means it is using more memory than it should or the memory isn't being freed?

bruno-nascimento commented 4 years ago

In what scenario did you see this? During a load test, or under normal use in production?

wrk -t16 -c500 -d30s -R100000 -s s*****.lua http://127.0.0.1:7000/login load test

The Argon2 algorithm uses a lot of memory by design --- the more memory required the higher the cost of calculating/verifying the hash

i shall change the default config values, thanks (and sorry for the open a issue when the problem was not reading the docs)

memory isn't being freed?

No, it takes some time but when the garbage collector wakes up, the memory is freed.