Open laurent22 opened 1 year ago
Here's what I've found so far:
ArrayBuffer
that should be faster. It's possible we're already using this, however — sjcl.decrypt
accepts a JSON string for cyphertext and sjcl
may be internally creating an ArrayBuffer. javax.crypto
in Android supports AES CCM). As such, it may make sense to eventually switch to AES GCM from AES CCM.
- If we want to switch to a native implementation...
How about libsodium?
How about libsodium?
I think libsodium
would also require a switch to AES GCM: https://github.com/jedisct1/libsodium/issues/739
sjcl.decrypt
accepts a JSON string for cyphertext andsjcl
may be internally creating an ArrayBuffer.
I skimmed the code and it looks like the files are encoded as base64 string first, then encrypted. I guess the conversion to base64 might slow down the process, and take extra storage.
Switching to a different lib would be for a future version since it's likely to be a lot of work.
For 2.12 either we find a way to make it fast again using sjcl, or we go back to AES-128. The current implementation may be more secure but unfortunately it's not usable.
Does changing the chunk size help?
It looks like larger chunk sizes generally result in faster decryption. Tests were run on an Android x86_64 API 33 (Android 14) emulator with 4 GiB of RAM. Encrypted data was 1 MiB and varied between randomly selected ASCII and unicode characters.
Averaging all trials (including the ASCII-only trials), SJCL1a
is on average about 1.2× faster than SJCL1b
. Similarly, a chunk size of 40,000 seems to be roughly 1.18× faster than a chunk size of 5,000.
Unicode tests only:
(Trial 1 and trial 2 were performed with SJCL1b).
Unicode and ASCII tests combined:
Interestingly, this conclusion is the opposite of what is mentioned in the code: https://github.com/laurent22/joplin/blob/90d75ce80e490cf5cabb4f3c1b0d9917d7d59872/packages/lib/services/e2ee/EncryptionService.ts#L58-L70
Thanks for these detailed tests but somehow 1.2x difference is not what some users (including myself) are observing. It's more like 5 or 10 times slower (or more) at least with certain resources. I'll see if I can replicate this slow down, so that it can be compared with the previous decryption method
5 or 10 times slower (or more) at least with certain resources
In that case, the decryption speed might be caused by memory usage (e.g. forcing the device to use swap instead of RAM). When running the tests above, roughly 87% of memory is being used on the emulator:
$ adb shell top -h
Tasks: 333 total, 1 running, 332 sleeping, 0 stopped, 0 zombie
Mem[||||||||||||||||||||||||||||||87.2] Swp[|| 3.4]
400%cpu 88%user 1%nice 4%sys 305%idle 0%iow 1%irq 0%sirq 1%host
PID USER %CPU [%MEM]CMDLINE
3328 u0_a169 90.0 14.3 net.cozic.joplin
580 system 0.0 6.9 system_server
1700 u0_a110 0.0 6.4 com.google.android.googlequicksearchbox:search
1244 u0_a104 0.0 6.0 com.google.android.gms.persistent
1575 u0_a104 0.0 5.9 com.google.android.gms
781 u0_a145 0.0 5.9 com.android.systemui
If the issue is low memory, I expect that a smaller chunk size would help, as less data would be loaded into memory at a given time. (E.g. maybe a chunk size of 4000?)
I'm retrying the above tests on an emulator with less RAM.
Thanks, so I guess the situation is that it will get faster as we increase the chunk size, until we fill up the RAM, at which point it will start swapping and becomes extremely slow. And I guess with AES-256 it tends to reach more quickly that limit.
I've just tried it and unfortunately I can't seem to replicate the same slow down I have on device. I think we're going to go back to AES-128 for 2.12 until we better understand the issue and how to tweak the encryption for performance.
until we fill up the RAM, at which point it will start swapping and becomes extremely slow.
Android doesn't use swap (except for something like zram or zswap on some devices), so if the RAM gets filled, the app will likely be just killed by the OS.
When running the tests above, roughly 87% of memory is being used on the emulator
but it's strange that the top
shows the joplin app only takes 14% of memory.
It seems to be slower when decrypting on mobile (tested on Android), maybe due to the recent change to AES-256.
Probably we need to test with resources for various sizes on the Android emulator, see how long it takes to decrypt. Changing the chunk size might help:
https://github.com/laurent22/joplin/blob/16d8a78d8a614a07886a99f696a6efaa76435f01/packages/lib/services/e2ee/EncryptionService.ts#L58
Ref: https://discourse.joplinapp.org/t/decryption-is-so-slow-on-android-2-minutes-per-1mb-note-that-it-makes-e2ee-unusable/32072/1