taoensso / nippy

The fastest serialization library for Clojure
https://www.taoensso.com/nippy
Eclipse Public License 1.0
1.04k stars 60 forks source link

Why would nippy/freeze hang on openjdk? #115

Closed tomconnors closed 5 years ago

tomconnors commented 5 years ago

On nippy 2.13.0 and 2.15.0-alpha9 (the only ones I've tested), the following works fine on my computer but hangs on my server: (nippy/freeze {:foo "bar"} {:password [:cached "foobar"]}). Note that there's no error message; it just hangs.

My computer is OSX w/ this java version: java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

The server is Ubuntu 18.04 w/ this java version: openjdk version "1.8.0_191" OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-0ubuntu0.18.04.1-b12) OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

Both are using clojure 1.10.0

I think the issue is due to a difference between Oracle java and openjdk, but the server's java install appears to have access to javax.crypto, confirmed with: /usr/bin/jrunscript -e 'print (javax.crypto.Cipher.getMaxAllowedKeyLength("RC5") >= 256);', and I don't know what else to check.

ptaoussanis commented 5 years ago

Hi Tom,

Does 2.13 also hang on your server?

First thing that came to mind: 2.15 introduced the use of GetInstanceStrong for the RNG, and that can block while waiting for sufficient entropy. Wouldn't by itself explain why your server is hanging indefinitely though.

tomconnors commented 5 years ago

Hi Peter,

Your initial suspicion seems to be correct. I suppose I didn't actually test 2.13 on the server - that one worked fine. When testing 2.15 again it blocked, but I think it would have eventually finished if I hadn't added a new entropy source.

I evaluated this at the repl, then went off to read about entropy on ec2 instances:

(time (nippy/freeze {:foo "bar"} {:password [:cached "foobar"]}))

While that was running, I checked available entropy with

cat /proc/sys/kernel/random/entropy_avail

The return was between 2 and ~65, dropping back down after reaching the mid sixties. I eventually learned that I could install the program "haveged" to increase entropy. With that program installed, that first form from the repl completed (after 40 minutes) and I tried nippy/freeze with the same and different passwords. The results were as the docs describe and the performance was good - ~1 second for the first time using a specific password and less than a millisecond for each additional time using that password.

If I wish to avoid installing haveged on the server, I can just use the AES-CBC encryptor, correct?

Thanks for getting back to me so quickly!

For posterity, here are a couple useful things I came across while looking into this:

ptaoussanis commented 5 years ago

Hi Tom, thanks for the follow-up.

If I wish to avoid installing haveged on the server, I can just use the AES-CBC encryptor, correct?

Unfortunately not, AES-CBC also needs entropy.

If you were to hypothetically switch to a non-blocking PRNG, your hanging problem would go away - but you'd be potentially introducing security vulnerabilities into your encrypted payloads.

My 2c would be to focus on trying to increase your server entropy. The resources you've linked above seem like a good start 👍

tomconnors commented 5 years ago

Thanks Peter, I appreciate the help.