Autoupload causes Nextcloud app to freeze and not respond after crashing is fixed

InfamousUser commented 3 years ago

Edit: Currently not testable as I cannot get to the point of uploads actually being queued due to a bug, this needs to be fixed first before being able to troubleshoot this further.

Steps to reproduce

Have QA version of Nextcloud from https://github.com/nextcloud/android/pull/7340
Have a lot of uploads set to be auto uploaded
Wait

Expected behaviour

File should get uploaded normally

Actual behaviour

Files get uploaded and the app constantly gets ANR dialogues and freezes the Android system interface half the time.

Can you reproduce this problem on https://try.nextcloud.com?

Please create a test demo account and see if this still happens there.
If yes, please open up a bug report
If not, please verify server setup and ask for help on forum

Environment data

Android version: 11

Device model: Google Pixel 2

Stock or customized system: stock

Nextcloud app version: 3.15.0

Nextcloud server version: 19.0.4

Reverse proxy:

Logs

Web server error log

Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Logcat logs (I think): [Nextcloud.txt](https://github.com/nextcloud/android/files/5567880/Nextcloud.txt)

NOTE: Be super sure to remove sensitive data like passwords, note that everybody can look here! You can use the Issue Template application to prefill some of the required information: https://apps.nextcloud.com/apps/issuetemplate

tobiasKaminsky commented 3 years ago

Do you see where and when it hangs? Can you give us a screenshot?

InfamousUser commented 3 years ago

I think it hangs when it attempts to autoupload. I don't need to be in the app, I can be in another app and I get the ANR dialogue. Otherwise I simply go to uploads menu and it freezes there if it's autouploading, if it isn't autouploading, then it might work normally.

When it says "App terminated" under the uploads, if I press "Failed/pending restart", it freezes, the Android interface freezes as well for 20~ seconds and then it displays ANR.

Nothing out of ordinary to screenshot; all looks normal, but when it starts to autoupload it's as if it gets overloaded and everything freezes while it processes.

tobiasKaminsky commented 3 years ago

When it says "App terminated" under the uploads, if I press "Failed/pending restart", it freezes, the Android interface freezes as well for 20~ seconds and then it displays ANR.

That is an important point, thanks!

tobiasKaminsky commented 3 years ago

How many failed items do you have?

InfamousUser commented 3 years ago

When it says "App terminated" under the uploads, if I press "Failed/pending restart", it freezes, the Android interface freezes as well for 20~ seconds and then it displays ANR.

That is an important point, thanks!

Oh?

How many failed items do you have?

37862

tobiasKaminsky commented 3 years ago

Failed/pending only reloads stored items from database and displays them. If you have >30k items this just takes too long. So this is somehow unrelated to AutoUpload, it just occurs due to the uploading of all old images.

You could trigger this also manually, when uploading 30k images.

InfamousUser commented 3 years ago

So this is somehow unrelated to AutoUpload, it just occurs due to the uploading of all old images.

That's what I thought, but then again I don't know much about programming so... This happened without adding upload existing files before though, I even reported a bug a while ago because Nextcloud was never uploading my stuff, only to realize it was never designed to do so when it was added as a feature and it never said it wouldn't. So this must have existed before that.

bardo commented 3 years ago

I had a similar issue when I enabled auto-upload on a folder with a few thousand pictures on a Pixel 3a. The app kept freezing and crashing, and after every restart it would start again from the beginning without looking for already synchronized files. I disabled power saving for the app as indicated in the manual, but it did not help, it was not able to run in the background. Power consumption was also extremely high while uploading.

As a workaround I forced the screen to stay on by enabling developer mode on Android and activating the corresponding option (you can also install Coffee for the same result), connected the phone to a USB cable, and let it synchronize with the app open. It took a long time but it worked.

Clearly, this isn't a good workaround for a non-technical user, but it can get the job done.

InfamousUser commented 3 years ago

Right, I think I did that one time with uploads menu open, if I didn't touch the app it wouldn't crash, but as soon as I tried scrolling or moving around it would hang, trying it multiple times caused a crash or non responsiveness dialogue iirc.

InfamousUser commented 3 years ago

Here is log for 3.14.1 freeze and crash in case it helps any. I scroll the upload menu slowly, it works for a couple of seconds, then around 15:19:40 it freezes for about 3-5 seconds, then automatically reopens with loading screen and whole Android is frozen for half a minute (system buttons on the botton don't work), then it unlocks and I get ANR dialogue.

Log_2020-12-05_15-21-30.txt

tobiasKaminsky commented 3 years ago

Judging from logcat, thanks for that, it seems indeed that the app uses too much memory and therefore Android tries to kill it. Maybe Pixel 2 is more restrictive…

InfamousUser commented 3 years ago

If it helps, in Android memory usage it says it uses an average of 26 MB and 156 MB maximum, if that's accurate. It should be able to use almost all RAM in foreground. I mean, I'm sure modern games take at least 1-2+ GB when running. There must be something specific that's tripping it up. Unless there's something that causes sudden spike in memory usage that doesn't show up in system data. It shouldn't naturally be using nearly as much as 1GB, but lack of memory shouldn't be reasonably happening.

tobiasKaminsky commented 3 years ago

Maybe the implementation of listing those files is sup-optimal and thus creates to much memory…

InfamousUser commented 3 years ago

Must be, it's been created that way. Since I've first seen it there was stuff wrong with autoupload, first it wouldn't upload, haven't been relying on it for years, or since introduction, then gave it another shot and it's been having these crashes/freezes for a year or two, at least it uploads stuff now. My previous phone failed around that time and didn't have anything backed up, lost all that was on it, the flash chip or bootloader failed for some reason.

I'd love to be able to use it reliably, as would the rest of the community I'm sure. Especially as an enterprise option. I don't understand how so many problems with one thing.

If you need any further data to get this fixed I'm available.

P.S. Not trying to flame, just being honest.

github-actions[bot] commented 3 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

oxc commented 3 years ago

Is there any further information required at this point?

InfamousUser commented 3 years ago

I don't know how to stop the bot from closing this. Convensional means of saying something doesn't work.

tobiasKaminsky commented 3 years ago

Is this only ANR (app not responding) when you want to list/refresh failed uploads? Or also if you click on restart failed uploads and then suspend app in background?

InfamousUser commented 3 years ago

Happens by itself in the background when I'm minding my own business (I assume when it autouploads), it happens if I restart failed uploads, it happens if I just open the uploads menu and look or scroll. Also perhaps important is that when I freshly open the app and go to uploads, it will upload a random small number of files successfully (lets say 3) and then it will freeze and then start not responding/stop responding (no more get uploaded after this point).

tobiasKaminsky commented 3 years ago

Do you still have those 32.000 failed uploads?

InfamousUser commented 3 years ago

20k now.

InfamousUser commented 3 years ago

I will try reinstalling the app, deleting the data on server and resyncing everything from scratch.

github-actions[bot] commented 3 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

InfamousUser commented 3 years ago

Original problems again. I have reinstalled the app, updated the server to the latest version, readded the autoupload, waited a few days, nothing got uploaded yet. I've removed and readded it again now.

Edit: Ok, there is "SSL initialization failed". I need to fix this first.

InfamousUser commented 3 years ago

App will not autoupload at all anymore. Opening a new issue for that first before this can progress.

github-actions[bot] commented 3 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

gomme600 commented 3 years ago

Having the same problem. Just opening the app is enough to crash it 90% of the time if a lot of uploads are waiting.

gomme600 commented 3 years ago

Can someone open this issue again please and stop the bot closing it?

Edit: Thanks

AndyScherzinger commented 3 years ago

You are welcome. Unfortunately I don't have the time to investigate this matter at the moment :/ And also @tobiasKaminsky is rather busy so this likely stays open for another while...

tobiasKaminsky commented 3 years ago

Having the same problem. Just opening the app is enough to crash it 90% of the time if a lot of uploads are waiting.

How many uploads do you have?

gomme600 commented 3 years ago

Over 2000

mauritsderuiter95 commented 3 years ago

I also have this issue for months now. I'm trying to move from Google Photos, but I have a lot of images. App keeps crashing. Once in a while I open the app again to try again, but it's just not usable this way.

Is the solution for now just keeping the app open and hitting "wait" every time the pop-up shows up telling me the app is not responding?

github-actions[bot] commented 2 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

InfamousUser commented 2 years ago

Brilliant.

bardo commented 2 years ago

Still happening, and getting worse.

After upgrading to Android 11, some apps changed their data directory due to scoped storage, forcing me to reconfigure the auto-upload for, e.g., chat app media. Most of the media is already synchronized, but the grand total is >5000 images.

This generated a lot of file conflicts and app crashes, so in some way I managed to disabled the synchronization and set the option to ignore the upload in case of file conflicts. I restarted the app, and now I can't access the auto-upload folder selection screen anymore. I get a blank screen and after a while Android proposes me to kill the app or wait. Waiting does not help.

I am under the impression that the whole auto-upload strategy needs to be re-evaluated to at least avoid blocking the app.

b-pfl commented 2 years ago

Still happening, and getting worse.

I am under the impression that the whole auto-upload strategy needs to be re-evaluated to at least avoid blocking the app.

I totally agree.

To add to this issue, I'd like to add observations from my current case: I am using the NC Android client on a Samsung Galaxy S9 and use the auto-upload to synchronize my images. Those images (~10500) are stored on an SD card and I tried syncing them ever since Owncloud / Nextcloud clients supported this.

It seems that there are multiple issue related to the auto-upload feature, which I just discovered when I had to re-install the NC client on my phone after repair.

Syncing a large amount of (existing) files seems to freeze the app and not respond properly
Even though Sync was switched on permanently over the last years (including the "include existing files" option since its existence), not all files have been uploaded. I did not find any common pattern yet regarding which files were uploaded and which were not.
There is no proper conflict mitigation happening in case of existing files duing auto-upload (see also https://github.com/nextcloud/android/issues/6911#issuecomment-900694464_): identical files (create date, size, checksum) should just be skipped, for the rest either the default option (overwrite, skip) should be performed or explicit user interaction is required.
In many cases of a conflict, the Android notification shows a conflict but when tapping on it, the conflict message cannot be regenerated.

b-pfl commented 2 years ago

Btw, this seems to be related to https://github.com/nextcloud/android/issues/5998?

InfamousUser commented 2 years ago

Lots of files is definitely related, but this is only a compounding issue.

github-actions[bot] commented 2 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

InfamousUser commented 2 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and see if the issue is solved, otherwise it makes no sense to close the issue. Thank you!

ajclements commented 2 years ago

I just encountered this today. Fresh install of server, version 22.0.0. Android 11, security update August 5; running on Pixel 4. App version 3.17.0.

m1435 commented 2 years ago

I can confirm this issue. I can't open the Autoupload Settings as it already causes the app to crash. Have retried installing the app - didn't help. Edit: Auto-uploading still works though, It's only the settings dialogue that is broken in my case.

Device: Oneplus 5T, Android 11/Lineage 18.1 App version 3.17.1 Server version 22.1.1.2

calderonth commented 2 years ago

Also seeing this issue. Opening the Auto-upload dialog hangs, dialog shows the app is hanging, forcing it to wait sometimes do brings the dialog up but unreliable and very slow.

m1435 commented 2 years ago

Also seeing this issue. Opening the Auto-upload dialog hangs, dialog shows the app is hanging, forcing it to wait sometimes do brings the dialog up but unreliable and very slow.

Seems to be working for me as well. I "only" have to hit "Wait" in the system dialogue like three or four times to get it working more or less reliably.

AlvaroBrey commented 2 years ago

@m1435 @calderonth Although likely caused by similar reasons, your issues are different from the original. The original was about ANRs when uploading tons of files at the same time, not when opening the Autoupload settings. Please open a separate issue for that.

noci2012 commented 2 years ago

On a new phone, enabling backup causes a crash and subsequent failure of the app to respond to anything.

`**** CAUSE OF ERROR ****

java.lang.NullPointerException: Attempt to read from field 'com.google.android.material.button.MaterialButton com.owncloud.android.databinding.BackupFragmentBinding.contactsDatepicker' on a null object reference at com.owncloud.android.ui.fragment.contactsbackup.BackupFragment$1.onPostExecute(BackupFragment.java:281) at com.owncloud.android.ui.fragment.contactsbackup.BackupFragment$1.onPostExecute(BackupFragment.java:252) at android.os.AsyncTask.finish(AsyncTask.java:771) at android.os.AsyncTask.access$900(AsyncTask.java:199) at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:788) at android.os.Handler.dispatchMessage(Handler.java:106) at android.os.Looper.loop(Looper.java:223) at android.app.ActivityThread.main(ActivityThread.java:7715) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:952)

**** APP INFORMATION **** ID: com.nextcloud.client Version: 30180190 Build flavor: generic

**** DEVICE INFORMATION **** Brand: Fairphone Device: FP4 Model: FP4 Id: RKQ1.210503.001 Product: FP4eea

**** FIRMWARE **** SDK: 30 Release: 11 Incremental: FP3N`

After this only popups with Nextcloud app stopped. (stop, report, info)... after info above report is shown, report of it from the phone fails. (this was copied, pasted in a mail ...).

LucImmerzeel commented 2 years ago

After reinstalling the app it worked normal when I didn't hide the default Auto upload's. I just didn't enable and didn't hide them. This worked for me. Maybe it works for someone else.

noci2012 commented 2 years ago

The issue i noticed with backup was on a new phone ==> a fresh install. Disabling backup again made the problems disappear. Phone is a FairPhone4, running Android 11

fuenfundachtzig commented 2 years ago

Same problem here, after hiding all automatically added "auto uploads" (11 folders in total; I would love to remove them rather than hiding them) I get a "app not responding" screen whenever I open the auto-upload menu item. If I select "wait", the app screen for configuring auto uploads will eventually show up after a minute or so.

(Could the delay arise because it still scans the folder even though they are hidden?)

icecore2 commented 2 years ago

Same issue here with 6k files and seems like around 5.5k duplicates. it hang right after the startup logo

nextcloud / android