Deadlock on memlock error

[PierreF]

It's possible to get a deadlock when calling Open() on an Enclave. This will happen if memory allocation or memory locking fail.

To Reproduce

I don't have simple step to reproduce. In our case I think we have multiple secrets for multiple PostgreSQL that we monitor (multiple == 20).

I do have the stacktrace and everything happen within github.com/awnumar/memguard:

• Our code call enclave.Open()
• This get the singleton Coffer and call View() on it
• We take lock in the View() function
• The View() function call NewBuffer() which Panic() on memcall.Lock(). I'm not sure which limit we hit, but anyway it could happen.
• The Panic() function will once more get the singleton Coffer and call Lock() on the Coffer
• It deadlock, it's the second time the same goroutine acquire the same lock.

goroutine 13222 [sync.Mutex.Lock, 2558 minutes]:
runtime.gopark(0xc001482d00?, 0x200000003?, 0x90?, 0xb5?, 0x4162b0?)
    /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc002c2b4c8 sp=0xc002c2b4a8 pc=0x43bc96
runtime.goparkunlock(...)
    /usr/local/go/src/runtime/proc.go:387
runtime.semacquire1(0xc000399084, 0x0?, 0x3, 0x1, 0x96?)
    /usr/local/go/src/runtime/sema.go:160 +0x20f fp=0xc002c2b530 sp=0xc002c2b4c8 pc=0x44d00f
sync.runtime_SemacquireMutex(0x22?, 0x0?, 0xc002c2b5c0?)
    /usr/local/go/src/runtime/sema.go:77 +0x26 fp=0xc002c2b568 sp=0xc002c2b530 pc=0x46b886
sync.(*Mutex).lockSlow(0xc000399080)
    /usr/local/go/src/sync/mutex.go:171 +0x165 fp=0xc002c2b5b8 sp=0xc002c2b568 pc=0x48d945
sync.(*Mutex).Lock(...)
    /usr/local/go/src/sync/mutex.go:90
github.com/awnumar/memguard/core.Purge.func1(0xc002c2b718)
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:23 +0x5a fp=0xc002c2b708 sp=0xc002c2b5b8 pc=0xf1e93a
github.com/awnumar/memguard/core.Purge()
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:51 +0x25 fp=0xc002c2b738 sp=0xc002c2b708 pc=0xf1e885
github.com/awnumar/memguard/core.Panic({0x26c0500, 0xc000a75b50})
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/exit.go:85 +0x25 fp=0xc002c2b758 sp=0xc002c2b738 pc=0xf1eda5
github.com/awnumar/memguard/core.NewBuffer(0x20)
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/buffer.go:73 +0x2d5 fp=0xc002c2b858 sp=0xc002c2b758 pc=0xf1bb55
github.com/awnumar/memguard/core.(*Coffer).View(0xc000399080)
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/coffer.go:86 +0xcc fp=0xc002c2b918 sp=0xc002c2b858 pc=0xf1d8ac
github.com/awnumar/memguard/core.Open(0xc00044d728)
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/core/enclave.go:111 +0x47 fp=0xc002c2b980 sp=0xc002c2b918 pc=0xf1e7a7
github.com/awnumar/memguard.(*Enclave).Open(0x256a6a0?)
    /go/pkg/mod/github.com/awnumar/memguard@v0.22.3/enclave.go:43 +0x25 fp=0xc002c2b9c8 sp=0xc002c2b980 pc=0xf223a5

System (please complete the following information):

• OS and Kernel Versions: Ubuntu 22.04, Linux 5.15.0
• Memguard Version: v0.22.3
• Go Version: go1.20.2
• Process Max locked memory: soft & hard are 65536 bytes