Android app crashes while loading large amount of notes locally on startup

[kiwipxl]

Describe the bug I've just migrated my 5600 notes (where did it all go wrong?) from Google Keep to Standard Notes. The notes have already been synched on the Android app, so when I start the app it needs to load 5600 notes from local storage. It takes roughly 5 minutes and can often crash during this process. I've tried turning off/on encrypted storage and it doesn't make a noticeable difference. I've had absolutely no problems on Desktop, just Android.

To Reproduce Steps to reproduce the behavior:

1. Have a large amount of notes (over 5000) in your account.
2. Open the Android app.
3. Crashing is pretty easy to reproduce. Wait for notes to decrypt. It should eventually crash, although not always. Try again if it doesn't crash.

Expected behavior No crashing.

Smartphone

• Device: Xiaomi Redmi Note 7
• Version: 3.5.15

[kiwipxl]

Hi, I've enabled bugsnag in a development build and have some more information on the crash.

The error is SIGABRT MainActivity. Here's the stacktrace: /system/lib64/libc.so:139628 abort /system/lib64/libart.so:4449928 art::Runtime::Abort(char const*) /system/lib64/libbase.so:36068 android::base::LogMessage::~LogMessage() /system/lib64/libart.so:2129672 art::gc::Verification::LogHeapCorruption(art::ObjPtr<art::mirror::Object>, art::MemberOffset, art::mirror::Object*, bool) const /system/lib64/libart.so:1719884 art::gc::collector::ConcurrentCopying::MarkNonMoving(art::mirror::Object*, art::mirror::Object*, art::MemberOffset) /system/lib64/libart.so:1728504 0x79d0eb7ff8 /system/lib64/libart.so:4659552 art::Thread::HandleScopeVisitRoots(art::RootVisitor*, int) /system/lib64/libart.so:4672580 void art::Thread::VisitRoots<false>(art::RootVisitor*) /system/lib64/libart.so:1726372 0x79d0eb77a4 /system/lib64/libart.so:921644 0x79d0df302c /data/app/com.standardnotes.dev-a1E01t9_SAX-dnqPth9mkA==/lib/arm64/libsodium-jni.so:7928 Java_org_libsodium_jni_SodiumJNI_crypto_1aead_1xchacha20poly1305_1ietf_1decrypt /data/app/com.standardnotes.dev-a1E01t9_SAX-dnqPth9mkA==/oat/arm64/base.odex:286544 0x79b652af50

(This stacktrace is for decrypting notes by the way). Looks like it could be a crash in the decryption. Could my device be running out of RAM?

Hope this helps.

[JaspalSuri]

Hi @kiwipxl,

I apologize for the late reply. Thank you so much for providing all of those details and enabling error reporting! I'll forward them over to the dev team for further review and hopefully they'll be able to resolve this type of crash.

[JaspalSuri]

Could my device be running out of RAM?

Presumably since the app is only downloading text it shouldn't, but in the event that it is, how much RAM does your device have? Different versions of the same model can have more than 3GB of RAM, according to GSMArena.

[lnolan5017]

I have a similar problem on my Nokia 6.1 Android Phone running Android 10 with 3G RAM, 32G internal storage, 128G SD card and my Samsung Galaxy Tab S2 running Android 7 with 3G RAM, 32G internal storage.

I recently imported over 1,000 notes from Evernote. I did it in batches and initially I had no problem with crashes. The crashes started happening when I got over 500 notes. I currently have 1345 notes and 77 tags.

Once I got over 1,000 notes, it pretty much crashed decrypting every other time it loaded. It always gets to at least 500 notes decrypted before it hangs. However, it can sometimes get as high as 1200 notes before it hangs. So it does not seem to be related to the data.

Once it hangs once on decrypting, it will just keep hanging on every subsequent load. If I clear the cache, it sometimes successfully recovers but not always. And the crash keeps on coming back.

I have tried all of the following without success:

• Turning off all data connections to eliminate any syncing issues while decrypting.
• Clearing the cache.
• Clearing the cache and data.
• Signing out and signing back in to force a full resync. The resync would hang sometimes as well.
• Uninstalling and reinstalling the app.

Even after uninstalling and reinstalling the app and successfully syncing, it would still hang while decrypting.

I have been forced to disable local encryption completely and now the app loads every time. However, it takes 17 seconds to load and see my notes on my phone. It only takes about 5 seconds to see the first notes I can work on if I enable local encryption.

I have local encryption enabled on the Windows 10 app and have no issues at all there.

I left Evernote because the latest release (version 10) also takes 17 seconds to load and then another 4 - 5 seconds to load a single short text note! I chose Standard Notes for the encryption, local storage and because it is open source.

I hope you can find and fix the issue so I can turn local encryption back on as 17 seconds to load is pretty inconvenient when I want to jot down a quick note.

[moughxyz]

Android operates with very tight memory conditions for our app for some reason. You will not have an issue with quantity for the large part, but only if you have some particular notes that are excessively large. If maintaining compatibility with the Android app is important, I recommend not exceeding 2MB per note (or even 1MB).

If you want I can lookup which notes are the largest (based on encrypted size) if you to to DM me your email.

What's the overall size of the backup file you imported into Standard Notes?

[lnolan5017]

I exported all my notes decrypted and unzipped them. The largest is 85K.

My largest import file was 568K.

I do have some very long note titles. Some were too long to unzip as it caused Error 0x80010135 Path too long in Windows. Not sure if that is anything but I will shorten them up just in case.

[lnolan5017]

In reviewing my decrypted zip file after export, I noticed I was missing over 100 notes in the Items\Note subdirectory. The notes did appear in the "Standard Notes Backup and Import File txt.txt" file. So this problem only affects the individual files generated for each note.

The notes that failed to export as an individual file all had one of the following reserved Windows File name characters in their title:

• asterix *
• greater than >
• less than <
• question mark ?
• pipe |

Removing these characters from the title resulted in the notes exporting correctly. Evernote adds the pipe character to some titles when adding a note using their web clipper so this was in quite a few of my notes.

In addition, the double quote " caused an error when unzipping. The note was exported but with the double quote character in the file name. This causes a "0x80070057 The parameter is incorrect" error. I use this character a fair amount for denoting inch measurements.

Finally, the forward / and backward \ slashes are removed while colon : is replaced by a space in the file name. I don't mind the space but I use forward slash in dates. Removing the / converts a date like 2014/12/01 into 20141202 in the file name. I would prefer to see a consistent replacement for all special characters whether that is a space or perhaps an underscore.

Once I corrected all of the offending titles I tried encrypting my local data again. Unfortunately, it did not fix the crashing issue.

Do you want me to submit this as a separate bug?

[moughxyz]

Do you want me to submit this as a separate bug?

Yes please! Can be created here: https://github.com/standardnotes/web

My largest import file was 568K.

Hmm, this is not bad at all. Definitely shouldn't be causing any issues. We recently fixed an issue for crashing related to encryption, but I hadn't explored decryption yet. I'll experiment more with small-size but frequent decryption to see if it's troublesome at all.

[lnolan5017]

Regarding the crash when the app is loading on my phone with decryption enabled, I should clarify that SN gets stuck decrypting and becomes unresponsive. This then generates a "Standard Notes is not responding" message from Android with the option to close or wait. I can still switch to the SN app, it just does not respond to any touch and the decrypting message is stuck on the number of notes it managed to decrypt before hanging.

Let me know if there is something you would like me to do on my end to help debug the issue. I have a development background but live more at the UX end of things.

[JaspalSuri]

Hi @lnolan5017, sorry for the delayed response.

Regarding the crash when the app is loading on my phone with decryption enabled, I should clarify that SN gets stuck decrypting and becomes unresponsive. This then generates a "Standard Notes is not responding" message from Android with the option to close or wait. I can still switch to the SN app, it just does not respond to any touch and the decrypting message is stuck on the number of notes it managed to decrypt before hanging.

Thank you for this info! I'll add it to our internal report. Just to clarify, by "decryption enabled" is device storage encryption enabled or disabled?

Let me know if there is something you would like me to do on my end to help debug the issue. I have a development background but live more at the UX end of things.

Thank you; you've helped us a lot so far. 🙂 If you have error reporting enabled via the Settings page on all of the apps across all of your devices, can reproduce the crash, and then let us know when it happened, that could be quite helpful.

[lnolan5017]

Hi @lnolan5017, sorry for the delayed response.

Regarding the crash when the app is loading on my phone with decryption enabled, I should clarify that SN gets stuck decrypting and becomes unresponsive. This then generates a "Standard Notes is not responding" message from Android with the option to close or wait. I can still switch to the SN app, it just does not respond to any touch and the decrypting message is stuck on the number of notes it managed to decrypt before hanging.

Thank you for this info! I'll add it to our internal report. Just to clarify, by "decryption enabled" is device storage encryption enabled or disabled?

Device storage encryption must be enabled for the crash to happen. I have been running on both my phone and tablet with no problem since I disabled device storage encryption.

Let me know if there is something you would like me to do on my end to help debug the issue. I have a development background but live more at the UX end of things.

Thank you; you've helped us a lot so far. 🙂 If you have error reporting enabled via the Settings page on all of the apps across all of your devices, can reproduce the crash, and then let us know when it happened, that could be quite helpful.

Do you need me to go through the Submit Feedback process when I get the Application not responding error? I generated the hang condition on my Samsung Galaxy Tab 2 today at 10:28am and the Application not responding error occurred at 10:33am.

[lnolan5017]

I have sent the feedback after reviewing the log. It seems libjsc.so cannot be found at 10:28:29.186. Hope that helps.

[JaspalSuri]

Hi @lnolan5017, thank you for clarifying and thanks for taking the time to do that! Can you let us know what time zone it occurred in or convert it to UTC?

[lnolan5017]

Hi @lnolan5017, thank you for clarifying and thanks for taking the time to do that! Can you let us know what time zone it occurred in or convert it to UTC?

Time zone is EST.

[JaspalSuri]

Thank you @lnolan5017; I will add that to my report. 🙂

[daveliao]

I'm also seeing the Android app crash during note decryption.

Using the Android app 3.6.24 on a non-rooted Android 10 device.

The decryption process makes it at MOST halfway-ish (1200/2100 notes). It usually crashes earlier than that. Happy to help troubleshoot this!

[JaspalSuri]

Happy to help troubleshoot this!

Thank you, @daveliao! We're looking into ways to optimize the decryption speed (at least in the Android app) and I will update this thread when a new beta with the optimizations is ready to test.

[andreas1107]

I've had a similar problem on Android if I tap anything while it's decrypting, it sometimes crashes. I only have around 550~ notes. It takes around 6-7 seconds from point of unlocking the app and notes to decrypt on a OnePlus 5.