Feeding tfcrypt with /dev/urandom in recent Kernel versions (4.8+)

[phantomcraft]

I use tfcrypt for generating large keys (up to 1024-bits).

Linux is using Chacha20 CSPRNG in /dev/urandom since version 4.8+:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=818e607b57c94ade9824dad63a96c2ea6b21baf3 http://lkml.iu.edu/hypermail/linux/kernel/1607.3/00275.html https://lwn.net/Articles/686033/

Chacha20 has a key space of 2^256 and a stream position counter of 2^64.

Does it make tfcrypt weaker when using this new /dev/urandom?

If an adversary could brute-force the entire Chacha20 key space and sweep across all 2^64 positions of its counter, he could guess the key much more easier instead of brute-forcing the entire Threefish key space excepting the tweak (it doesn't add bits of security because once knowing the correct key, calculating a tweak is easy).

[strlcat]

Indeed, I think it could. And older Linux kernels happen to use sha1 IIRC, which is bad. For really strong and/or longterm keys I do use /dev/random with '-r' option. That's also true for any valuable long term storage encryption.

[phantomcraft]

It was a very bad idea to include Chacha20 in urandom and the problem with /dev/random is that it will block sometimes and can crash the programs are using it if entropy is low in the system. I have been using the package rng-tools (rng-tools5 in Debian) to boost /dev/random with use of the hardware RNG to gain entropy.

I generate my keys this way: sksum -D 1024 -W -l 1M /dev/random

[strlcat]

Why? I think situation was worse in older kernels and I happy the devs did a move from '90s practice anyway. urandom is probably not what you want for a secure keygen source. For anything else (incl. ordinary keys), it suffice even with old sha1. Bruteforcing ChaCha20 should be alot harder than it was before, and ciphers like Threefish indeed do require TRNG if you want really secure keys.

[strlcat]

tfcrypt includes another handy mode to generate keys, and may generate keys of arbitrary length this way: tfcrypt -r /dev/random -R 128 key.bin. This will give you a 128 byte key, and /dev/random will be polled until the buffer will be fed up. Requiring 1 Megabyte from /dev/random is wasteful, and gives you little improvement over standard key length. (I have no hardware RNG so you may imagine how slow /dev/random is)

[phantomcraft]

@lynxlynx

I know it seems weird to generate a key that way but I don't trust /dev/random for generating a single 128B key file. It seems the /dev/random continues to use sha1: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/char/random.c?h=v4.20.6#n317

Both random and urandom uses a CSPRNG which certainly has less security margin than 1024 bits. For generating a 1024 bits key which is 128B, I have my doubts if /dev/random CSPRNG can be manipulated to do a key recovery attack is less than 2^1024 time. Correct me if I'm wrong.

I installed rng-tools5 in Debian which provides the char device /dev/hwrng for getting random numbers from hardware RNG which can't be exploited for any key recovering attack, although I don't trust it very much to generate a small byte sequence. I thinking on re-encrypting all my stuff that I have encrypted with threefish using keys processed from /dev/hwrng source. /dev/hwrng is very slow but is a true RNG.

[strlcat]

At first I thought that SHA1 is used just to add pseudorandom material to truly random pool, which is harmless. However the way SHA1 is used for /dev/random in Linux kernel is dangerous: the output is SHA1 biased, and may produce collisions (still unlikely to face this, until good attack vectors like that was with MD5 will be discovered). That's only if I got the source correctly - I am not a current kernel expert.

I wonder why people still use SHA1 hash functions up to today. I can understand usage in git for example - it was a historic mistake which made it into millions of repositories worldwide and can't just easily be fixed. But occasions like that just should be condemned.

I personally thought they got that part fixed along with introduction of ChaCha20. However, they're not. That's bad. And that's hilarious, but /dev/urandom now is more secure (if reseeded from random data, I lost the track what pool actually gets used) in my eyes. And fast. Someone should fix /dev/random output not to use SHA1 at all (it's ok however to bias it back with SHA1 or whatever modern thing - why not use ChaCha20 again? pseudorandom + random is still random BTW), or, at least file a kernel bug. If you could - that would be good, I will join then. I am myself not into this business.

Back to your questions. If you do hope that no /dev/random nor /dev/urandom can be trusted (one being still hilariously biased by SHA1 -- with collisions existing for SHA1 since 2017), you should generate keys on your own input. You do have hwrng (I hope raw one) - that's good for your healthy paranoia. What do other filthy peasants should do? Well, if they don't mind the quality, using plain but long passwords is still OK, even if nr.of bits is lower than result key bits. Keyfiles do too, if keyfile is unique thing like PNG image. There are way too many options and imagination will help you. For me - nothing changes, except that I will trust urandom more since that I have latest kernel.

If you feel you have more questions to ask, just reopen this bug. There is nothing to fix in tfcrypt I believe.

[phantomcraft]

I found an interesting project: https://github.com/sandy-harris/maxwell | https://github.com/sandy-harris/maxwell/blob/master/doc/Maxwell.pdf

Is an entropy gathering daemon which feeds on CPU timings, it's a True RNG, as I tested here, runs fine without root privileges, can be compiled statically and generate random data: ./maxwell -s -p 6 -h 8 | sksum -D 1024 -W > /path/to/key_file

tfcrypt with such a TRNG would be a very good idea because it would be free of external CSPRNGs like that of /dev/(u)random, running on an old or low entropy system would not be a problem.

[strlcat]

The method described in paper is good only for busy systems. This means idle routers are out of luck, and here is why. I simply take my old AR9331 chap, a TL-MR3020 pocket router, which by default runs completely idle. Then I build tfcrypt and copy to it and run the benchmark several times:

# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262132 (255.99K) B/s, time 2.0001s.
# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262124 (255.98K) B/s, time 2.0001s.
# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262132 (255.99K) B/s, time 2.0001s.
# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262132 (255.99K) B/s, time 2.0001s.
# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262131 (255.99K) B/s, time 2.0001s. 
# execvp ./tfcrypt tfbench 2
tfbench: doing XTS benchmark for 2.0000 seconds ... done!
tfbench XTS benchmark results:
encrypted 589824 (576.00K) bytes, avg. speed 262131 (255.99K) B/s, time 2.0001s.

Look at the speed numbers. The numbers are almost the same, and even match sometimes. The time measured is not such good shown here, but both # of processed bytes and measured speed is almost identical. This may indicate that signal arrives at precise intervals. Same happens with my own cipher, which is waay too faster on that thing (about 6,3MB/sec), or any other modern ARX cipher. But the result is same - numbers often collide. In contrast, on my idling desktop there is no such miracle - the numbers are always different :-)

Of course measured nanoseconds will be always drifting, even on such truly idling thing, but extracting entropy from such drift is not a good idea. This drift simply may contain repeatable patterns (at certain moments), and extrapolating it into megabytes of derived random numbers from TF, it may end up in a true disaster like Debian OpenSSL fiasco.

I would not recommend this way.

[phantomcraft]

I also want to test tfcrypt in that way. Is there a way to simulate such a idle system on VirtualBox or QEMU/KVM? I've only a desktop system.

[strlcat]

I doubt that's possible, but you may try to build a kernel with fixed clocking, then boot it in QEMU with basic hardware set with minimal initramfs which will spawn a busybox shell for you, and it will contain tfcrypt binary. This is a mips embedded Linux, it is possible that it does not have too many random/wakeup sources (mainly, only NIC activity) to deal with. It even slow at it's urandom to initialize. Aside from that, I would not still rely on clock drift as a random source anyway, because of bias reasons I described in my previous comment.

[phantomcraft]

I understand, but as I read even a unbiased TRNG should not be used directly in generating key material, a (cryptographically secure) randomness extractor should be applied on bit stream before, being the most simple and efficient method a secure hashing of it.

A very biassed entropy source could be used for generating a key assuming the bit stream is sufficiently long and not taken from a predictably source: https://crypto.stackexchange.com/questions/48629/extracting-randomness-from-highly-biased-rng

Following this with biased stream with 2 bit per byte, calculating a 1024 bit key extraction should be very simple: (1024*8)/2 = 4096 bits stream at least to be securely hashed.

[strlcat]

I assume you understand that size of truly random sequence matters. If you're out of picture, I'll explain it with a very simple case in hand: imagine you have a coin and flip it once per second. Then you hash/encrypt this tiny truly random result. How many unique streams you will get from a single coin flip? Exactly two. If you have 8 bits per second, the you should wait a very long time to collect as many random flips as possible, and of course to not show them to anyone before use.

Biased output means that randomness falls apart - it contains patterns or holes predictable enough to shrink/compress the possibility space. One may analyze even hashed output and find out that sometime, someday, the sequences do repeat. Even if not, by chance, one may get the impression when such repeat did occured once, then try to brute force the space to confirm his finding. Given that today's cryptographic stuff is on public, it's even easier to analyze existing implementations for such faults.

At least, /dev/random and /dev/urandom are controlled by the hardware noise sources, and the hardware sets differ very wildly between Linux machines.

If you're truly paranoid in a good words, get yourself educated about electronics and construct a simple TRNG generator based on a Zener diode breakdown noise. There are ready to use projects available too, because connecting it's output to PC maybe a pain.