search

nextcloud / android

📱 Nextcloud Android app
https://play.google.com/store/apps/details?id=com.nextcloud.client
GNU General Public License v2.0
4.32k stars 1.78k forks source link

Auto upload uploads completing then looping (repeating) #11263

Open patrick7 opened 1 year ago

patrick7 commented 1 year ago

⚠️ Before posting ⚠️

Steps to reproduce

  1. Install the android app from play store
  2. Connect to the server
  3. Set up an auto upload for camera

The camera folder contains around 1700 files

Expected behaviour

The 1700 files are uploaded and no more pending files are shown

Actual behaviour

The client shows more than 3'000 files. After the really existing 1700 files are uploaded (I see PUT/MOVE/HEADs), the client's counter still decreases and I see HEADs in my webserver's logs.

After the process is reaching almost 0, the counter increases again to a few thousands. I still see the HEADs on my webserver, but no more files are uploaded.

Also, if I refresh the uploads screen by swiping down, the counter increases.

Android version

13

Device brand and model

Pixel 6 Pro

Stock or custom OS?

Stock

Nextcloud android app version

3.23.1

Nextcloud server version

Nextcloud Hub 3 (25.0.2)

Using a reverse proxy?

Yes

Android logs

No response

Server error logs

No response

Additional information

No response

TillW99 commented 1 year ago

I have around the same bug behavior, except that there are about 800files... Stacktrace:

[Removed as it was unrelated to this issue]

App information

Device information

Firmware

patrick7 commented 1 year ago

@TillW99 Are you running any caching? After I ran into this issue yesterday, I've installed a REDIS cache and I think this might have (partially) solved the problem. The sync-process was still showing around 5000 files (instead of 1700) but it stopped increasing and decreased to 0 and the sync seems to be OK now.

I'll have to do some further testing.

TillW99 commented 1 year ago

No, i dont think so - i changed nothing in my Nextcloud config. But since November the upload does not work properly, the app says that the files have been uploaded, but this is not true. The files are arriving in the cloud, when i keep the app on. Otherwise it wouldnt

lgrawet commented 1 year ago

I had the same problem then I decided to delete app storage and start from scratch. After having configured back auto-upload, nothing happens and I have no clue why. Upload history remains empty, no counter.

I'm running Android 12 / Nextcloud App 3.23.1 / Nextcloud 25.0.2

lgrawet commented 1 year ago

OK, it looks like it took quite some time to index. I now have 16 402 files in queue, currently not decreasing

lgrawet commented 1 year ago

And now it is looping again, draining the battery

ccman32 commented 1 year ago

Same problem for me. The whole auto upload feature is basically unusable for me. As soon as I set it up for at least one folder that contains 1000+ files, the app becomes unresponsive because it keeps trying to upload every single file all over again. It even continues doing this after I have disabled the auto upload for every folder, so I can either fully reset or reinstall the app afterwards or I won't be able to use it anymore.

I am running Android 13, Nextcloud App version 3.24.1 and Nextcloud 25.0.3 on my server.

BLu85 commented 1 year ago

Different behaviour here:

Fresh app installation v3.24.1 (server is v 25.0.3).

Set up auto upload with:

The remote folder contains +5k pics. The local folder contains the same pics and few more that should be auto uploaded.

Phone has been charging for the entire night.

Actual result: upload section is empty, zero files uploaded. Expected result: files in the local folder that are not in the remote folder should have been uploaded.

Also experiencing battery drain with Nextcloud app consumption level up to 30%.

p.s. everything was working fine before I updated the server from version 24.0.x to 25.0.x

lgrawet commented 1 year ago

I was expecting more reaction from Nextcloud team on this issue. I had no other choice than to switch to Syncthing unfortunately.

arnowelzel commented 1 year ago

Same strange experience here. Fresh install on a Fairphone 4 with Android 12 and Nextcloud 3.24.1.

Server is Nextcloud 25.0.4 with APCu memory cache, Redis for file locking and client push aka "Files High Performance Backend". The server has been working without any issues and around 20-60 active users for many years now.

On the phone I have around 3500 files.

Upload starts with around 3200 files and sends hundrets of HEAD requests to the server. After it had counted down to about 1800 files it started increasing(!) again up to 3400 files and again sending about 5-10 HEAD requests per second to the sever. Then it counts down to about 1600 files and stops.

Then as soon as I refresh the "Upload" view manually in the app it increases(!) again to about 3480 files and continues sending HEAD requests to the server for the same files which already have been checked. Now the top 4 entries in the list remain the same while other files seem to get processed.

Then it stops with 8 files. And the same again: I pull to refresh the view and it goes up to 3480 again and it starts sending hundrets of HEAD requests.

Something is really wrong here and this way the client is totally unusable for me.

endurlanion commented 1 year ago

Having the same issue .. shows 27k photos, permissions are given and nothing is uploaded.. in addition the app makes the phone very slow. On server side i am running nextcloud from snap ootb

arnowelzel commented 1 year ago

Eventually I switched to FolderSync (which supports Nextcloud servers as target out of the box) as I can not wait until this problem has been solved. FolderSync works fine and also has an auto-upload feature for changes. In addition in FolderSync it only takes a few seconds to get the list of the existing 3500 files from Nextcloud using a single PROPFIND request and compare them with the folder contents. FolderSync I don't understand, why the Nextcloud client sends a HEAD request for every single file. Even when 10-15 requests per second are possible this would still take at least 4 to 6 minutes(!) to check the existing files.

luxzg commented 1 year ago

I've installed NC26 and current NC Android app, setup auto upload for camera folder and one small 2-image test folder. Test folder was uploaded immediately. Camera folders not so much. At some point video portion was triggered by unknown powers, and files uploaded just fine after that. But that was 6 days ago, and I have 2 new videos sitting on my phone, no luck triggering upload again. Image portion of the Camera folder is stuck no matter what I do. It just won't queue any files for upload. Tried setting up folder manually, still no queue in uploads.

I thank above commenters for suggesting 3rd party tools. But Syncthing needs couple workarounds, uploading to external folders, fixing permissions, triggering scan on server side, and has reports of corruption. Foldersync is propriey, ad-infused, can also cause corrupted files.

My workaround may be syncing to OneDrive then importing to NC.

Or I'll just manually upload my stuff. Not nice. But obviously not much else is possible...

arnowelzel commented 1 year ago

FolderSync is an app which only needs the source and destination configured. The source will be the camera folder on the device and the destination can be a folder on the Nextcloud server. It can also monitor changes and automatically upload new files.

luxzg commented 1 year ago

I've found another app called "Zpush" ( Play store link ). Since I use TOTP / 2FA I've created app password, and it's already syncing my photos. App was recently updated, so it's alive and under development. I haven't had time to explore much more, but it has options like when to run (on WiFi, when charging, only if host is reachable)

Btw I've found it thanks to this Reddit post

App password

To get app password click your avatar in top-right when logged in, Settings, Security, or go here: <yourNextcloudURL>/index.php/settings/user/security Scroll to "Devices & sessions", enter new app name eg "webdav", and you will get username and password to use for the app. Once you save the credentials somewhere safe, click "Done" to save them.

Inside Zpush app

Then open Zpush app, and click bih "+" icon to add a new job.

Settings for job name inside Zpush app:

Source section

Local directory to push: primary:DCIM/Camera (no filters, no deleting) Include all subdirectories and files (ON)

Target section

Target filename (left at default) Driver: WebDAV Host ip or Hostname: 192.168.192.168:80 SSL: (OFF) haven't set it up yet Username: Password: <from app passwords screen eg 1234-abcd-1234-dfgh-5678-efgh> Target path: /remote.php/dav/files/myusername/InstantUpload/Camera

Hope this helps someone. And hopefully app stays alive. If anyone needs help with this app just reply to my post and I'll try to help you by comparing our configs or whatever else I can do as an end user.

Oh, btw, Android 12 and NC26. (and it's at file 700 of 7000, I'll edit post inafew days with how it fares)

geez0x1 commented 1 year ago

I've also started experiencing issues recently. Tons of pictures either on "Waiting to upload", or under "Failed/pending restart" with "Sync conflict, please resolve manually". Pressing the "restart failed" thing does not resolve it.

It does seem that sometimes, something does upload (random). In fact I've manually checked a few cases and so far it seems all files have actually been uploaded. And, taking a new picture just now results in a successful upload.

Judging from the number of people in this thread (and duplicate(s)) already this seems to be a recent regression.

james3126 commented 1 year ago

I seem to suddenly be getting this same fault. Got a few files stuck on waiting for upload. I can't retry, cancel, etc... The uploads tab is essentially frozen.

Seems most photos are still uploading in the background as I have my most recent photos in my NC gallery.

BLu85 commented 1 year ago

I seem to suddenly be getting this same fault. Got a few files stuck on waiting for upload. I can't retry, cancel, etc... The uploads tab is essentially frozen.

Seems most photos are still uploading in the background as I have my most recent photos in my NC gallery.

It sounds like your issue is slightly different from what is posted here and you are in a luckier position as you get file uploaded :D We don't get files uploaded. Different useres experienced the files to be queued in the upload section, but they are not uploaded to the server. I don't understand if this issue has been considered from the Nextcloud team, as I don't see any feedback on this case.

TimoKramer commented 1 year ago

Sync conflict. Please resolve manually is an error that I see in my degoogled phone since months if not years. I can not upload any new file to my nextcloud server. It happens with auto-upload and manual upload.

Current version server: 25.0.7 (but at least a year of updates back it was the same problem) Current version client: 3.24.2 ( " )

RokkuCode commented 1 year ago

This has never worked well. I am using this app since day one and it is broken since day one. Sync conflicts are the only function that works well. It reports a sync conflict even on completely identical files. the only way to fix this is to delete it completely, remove all data and sync fresh from server. than do not use the auto upload function because everything is broken.

jobe451 commented 4 weeks ago

I just tried these days nextcloud. My main motivation to get photos and files synced from my Android. I had exactly this problem. 13K files hardly any files arrive and the process keps starting from new again. When I found this issue that was not touched for a year, it got clear to get rid of NC. And search for something else.

patrick7 commented 4 weeks ago

I now use syncthing for the sync part. For the file browser you can choose whatever you want :-)