WebDav sync unreliable, ARCHIVE.json not synced properly

[BuggyHuman]

Your Environment

• Version used: 9.0.7
• Operating System and version:
  • Windows 10
  • Linux Mint 21.1
  • Android 13
• WebDav sync with mailbox.org, no compression, no encryption

Expected Behavior

ARCHIVE.json should be uploaded reliably.

Current Behavior

There seems to be an issue with uploading the archive data. About Once a week it fails to upload the archive data properly. When this happens all devices fail to sync, except the device that uploaded the broken (?) archive data. When I sync from the device that uploaded the data there seem to be no issues. The only way to fix this issue without dataloss is to delete the MAIN.json and the ARCHIVE.json in the cloud and resync.

Before the error appears the app asks me, if i want to use remote data or upload local data. When I choose remote data I get the following error.

I've been observing this issue since Version 9. I did not have this issue in Version 8.

Steps to Reproduce (for bugs)

I don't know a way to reproduce the bug.

Can you reproduce this reliably?

I am not able to reproduce the error reliably. It seems to happen randomly.

Console Output

I am dumb and fixed the cloud data before taking a look at the console. If the error happens again i will take a look at it.

Error Log (Desktop only)

Same as Console Output

[github-actions[bot]]

Thank you very much for opening up this issue! I am currently a bit overwhelmed by the many requests that arrive each week, so please forgive me, if I fail to respond personally. I am still very likely to at least skim read your request and I'll probably try to fix all (real) bugs if possible and I will likely review every single PR being made (please, give me a heads up if you intent to do so) and I will try to work on popular requests (please upvote via thumbs up on the original issue) whenever possible, but trying to respond to every single issue over the last years has been kind of draining and I need to adjust my approach for this project to remain fun for me and to make any progress with actually coding new stuff. Thanks for your understanding!

[github-actions[bot]]

Hello there BuggyHuman! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

For more open ended discussions and/or specific questions, please visit the discussions page. 💖

[spotman]

Same error on Android app and web app in browser. Desktop app is working fine and was used to upload actual data to webdav. Error still exists even after forced sync from desktop app to an empty webdav directory. No errors on webdav backend, MAIN.json and ARCHIVE.json are stored successfully (json is valid).

Screenshots are below.

[mwaterbu]

Also hitting this between Android and Linux desktop app.

[johannesjo]

@spotman & @mwaterbu what provider are you using?

It looks like that the rev returned on upload is not the same it returns for subsequent get requests. @spotman 's case looks fixable by a simple replace, but not sure if we can generalize it somehow.

I would much appreciate your help with this! How do do the response headers look like for upload, download and the HEAD request look like? What values are present for etag, oc-etag and last-modified?

[mwaterbu]

I am just using WebDAV sync to a Synology NAS on my local network.

[sweetman-zz]

Hello, I have the same problem: webdav sync works on Windows 10 but get same error on Android. I use a qnap nas. On the same nas I use webdav to sync my 10gb joplin notes succedfully. Hth Tony

[sweetman-zz]

I made it to work. My solution is to set the base URL with the main super-productivity directory and then specify a subdir inside this directory. Example: base URL: https://mycloud.ext:port_number/super-productivity Sync target: subdir

My shared folder structure:

/super-productivity/subdir

Don't know why, but this way it works. HTH tony

[culinia]

the solution mentioned by sweetman-zz didnt work for me.

seems my problem had been caused by different versions on android and linux mint: on linux i had the .deb version installed which obviously was older than the android version from f-droid.org.

=> on linux (mint debian edition 6) i now use the version from snap plus on both devices (fairphone 2 with lineage os 18.1) the compression feature activated. since then it finally syncs in both directions without any error messages, yeah!

update:

syncing from remote with mobile phone works only if i delete the app's entire storage (not only the cache) on my phone first. if i dont do that the above shared error messages that were already shared by the others here, persist. maybe thats a helpful hint for the developers how to solve the prob. f.ex. if you choose to use the remote file (which obviously doesnt seem to be broken unlike claimed, otherwise it couldnt be imported successfully into a just installed client) then there needs to be an automated script in the background erasing the local database before downloading the remote one.

update 2: the above is unfortunately not reproduceable. at least not with my devices. meanwhile i tried the version from gplay store, too after i run into the error message again when trying to sync back from phone to desktop. although the gplay version was upladed there a few months later, (august vs april 2023), this didnt solve the problem.

so under the bottom line, if you dont want to use any data collecting third party sync service or setup an owncloud/nextcloud server, there's at the moment no working bidirectional possibility to sync your todolists. really frustrating. makes me almost look for another, comparable software, but these are very difficult to find if they exist at all yet...

so lets all please donate for the further development of this amazing tool, so Johannes could can finally give even more focus to this project than up to now or eventually hire a second developer.

[sweetman-zz]

I made it to work. My solution is to set the base URL with the main super-productivity directory and then specify a subdir inside this directory. Example: base URL: https://mycloud.ext:port_number/super-productivity Sync target: subdir

My shared folder structure:

/super-productivity/subdir

Don't know why, but this way it works. HTH tony

Never mind. After updating the android app and the windows 10 exe application, it doesn't work anymore. Even if I use the correct paths, when I sync the android app a pop-up asking to resolve conflict by using local or remote data. If I select remote I get the "last sync incomplete" pop-up. No way to make it work.

Please note that on pc desktop I use data stored on NAS on my local network. Webdav base URL: http://192.168.0.2:port1/super-productivity Webdav sync folder: mysyncdir/

On android I have this setup: Webdav base URL: https://mycloudserver:port2/super-productivity Webdav sync folder: mysyncdir/

Could it be the port number? I have two ports because port1 is on HTTP (locally I didn't installed any certificate) and port2 is on HTTPS.

What else can I try? TIA tony

[sweetman-zz]

I'm pretty sure the sync worked between pc and android with an old SP version (10.2) of the app. I tried to reinstall them but it doesn't work any more. Tony

[johannesjo]

I made some adjustments to handle the etags that start with the W/ used to indicate weak etags. Not sure if this will fix the problem for everybody. I will make a release with that change later today.

I also noticed that a recent change leads to paths handled differently so please check if adding a trailing slash to your baseUrl does help in some way..

[sweetman-zz]

Hi Johannes, I installed SP on another windows 10 pc, outside my local network and it works using this setup:

• Base URL: https://mycloud.com:SSLPORT/super-productivity/
• shared folder: shared-folder/ But on android it doesn't work: I get the message to use local or remote data and when I select remote data, a pop us says sync is incomplete. Using sync 10.0.9 on every device, now. Tony

[BuggyHuman]

I have found a way to reproduce the issue. It can be reproduced on any Device. I tested it on 3 Devices. Here are the steps:

1. On Device 1: Change some data and close Super Productivity before it can sync.
2. On Device 2: Change data and sync it.
3. Go back to Device 1 and try to sync data. You now receive the error mentioned above.

Unlike what i wrote in my first post Device 3 still works afterwards.

[johannesjo]

Did v10.0.10 fix the issue for some of you?

[sweetman-zz]

Installed 10.0.10 on both pc and android: same problem, once I try to sync.

[johannesjo]

Testing this with nextcloud all works fine. As mentioned above I would need more info about how the revs look like, since I can't realistically test this with all available webdav providers :)

[spotman]

Did v10.0.10 fix the issue for some of you?

I can confirm that sync is working now, but only after removing files from WebDAV storage and uploading them via desktop app.

Thanks for the fix! I've tried to get rid of weak etags by myself but apache (mod_dav + mod_dav_fs) has no configuration for this.

[sweetman-zz]

I installed 10.0.11 and followed "spotman" suggestion and... it works! Tony

[culinia]

yes, now it finally works!

the steps I did:

1) update desktop client via snap refresh superproductivity --beta (without the --beta parameter it doesnt update)

2) I made sure that on the phone the version from google playstore is installed.

3) deleted all local stored app related data from the phone

4) deleted the old remote file from the webdav device

5) uploaded the local version stored in the desktop client to the server

6) adding the webdav sync infos into the phone client. (i did a fresh install of that one in my case)

7) download the remote version on the phone

8) as a test change any entity within the app on the phone and sync them to the server

9) click sync button in the desktop client, choose remote file

10) change sth in the desktop file, sync to the server, then repeat step 9 on the phone.

10) shouting jipeeee and doing a jump of joy! <3