johannesjo / super-productivity

Super Productivity is an advanced todo list app with integrated Timeboxing and time tracking capabilities. It also comes with integrations for Jira, Gitlab, GitHub and Open Project.
http://super-productivity.com
MIT License
10.88k stars 865 forks source link

Unable to Sync with Dropbox After Restart #3288

Open FlysonBot opened 1 month ago

FlysonBot commented 1 month ago

Your Environment

Expected Behavior

I should be able to sync with Dropbox permenantly.

Current Behavior

After I configure the syncing, it would sync at that time. But once I close the app (close the website on chrome or force stop on android), when I open up again it wouldn't be able to sync. It said something like (translated to English) initial sync failed, cannot verify identity, and when I sync manually it said cannot generate access token from verification code.

Steps to Reproduce (for bugs)

  1. Open the website / app
  2. Configure the syncing with dropbox
  3. Close website / Terminate the app
  4. Reopen the website / app

Can you reproduce this reliably?

Yes, 100% of the time.

github-actions[bot] commented 1 month ago

Thank you very much for opening up this issue! I am currently a bit overwhelmed by the many requests that arrive each week, so please forgive me, if I fail to respond personally. I am still very likely to at least skim read your request and I'll probably try to fix all (real) bugs if possible and I will likely review every single PR being made (please, give me a heads up if you intent to do so) and I will try to work on popular requests (please upvote via thumbs up on the original issue) whenever possible, but trying to respond to every single issue over the last years has been kind of draining and I need to adjust my approach for this project to remain fun for me and to make any progress with actually coding new stuff. Thanks for your understanding!

FlysonBot commented 1 month ago

Additionally, if I'm using the android app, after it failed to sync, when it show the window that ask me for the verification code again, the botton that is supposed to open up the verification link doesn't work. It work on the online version though. On android it only work the first time when I configure it.

dobsche commented 1 month ago

I can relate to that and would really appreciate a fix

FlysonBot commented 1 month ago

Actually, I just found with the online version that even if I don't restart, after a while the syncing will fail and it say (translate to English) DropBox is providing an invalid access token [Botton: Change the Token]. I don't know about the Android because I don't have it currently installed.

johannesjo commented 1 month ago

I am unable to reproduce this. Configuring dropbox from the browser (latest chrome) works fine for me and so does starting in the app (android pixel 5). What am I missing?

FlysonBot commented 1 month ago

Hmm. I think my claim earlier was wrong. After a while, I opened the app on my phone browser again but the sync magically starts working. And no matter how many times I close the browser it still will work. Untill I go to my desktop browser and configured syncing from there and sync once, the syncing on the phone stop working. And if I don't sync on my phone browser, no matter how many times I close the desktop browser the destop can still sync. So I guess the problem is not so much with the app but maybe something with dropbox. But I would still expect I can sync from both device so I can use it on both devices. And thank you for all your hard work and the quick response, the app is excellent and I'd use it for a long time, very appreciated and love it!

FlysonBot commented 1 month ago

After a while of single device syncing on my laptop on chrome, I got "Dropbox is providing an invalid token [botton: change token]" error (translate to English).

johannesjo commented 1 month ago

Were you able to change it? I think this might happen if the token was not used in a while, but haven't experienced it myself so far.

FlysonBot commented 1 month ago

So right now there's 2 issue:

  1. Cannot sync on multiple devices. If I try, it will sync, but the app/website is restarted on either device, that device cannot sync. For example, if I have the website opened on device 1 and device 2 and both configured syncing, if I now close and reopened the website on device 1, it'll failed to sync. While device 2 can still sync untill it is closed and reopened as well.
  2. Cannot idle for a long time. I set the auto sync time to 5 minutes, and it works fine. If I'm only using 1 device, it work as long as the computer is on. However, once I close the laptop and go for lunch (which is like 30 minutes and so), when I come back it'll failed to sync. So I guess the token expired once it is not used for certain amount of time.

Yes, I can change the token, every morning and every times after lunch and dinner, and it'll work. But I wish the experience can be improved by resolving the multi-device syncing issue and by avoiding the token to be expired. Thanks!

johannesjo commented 1 month ago

Could you maybe provide logs for all the steps of the problems you describe?

dariusz-f commented 1 month ago

I have similar issue. Is the Auth Code for Dropbox have to be the same on all synced devices, or generated for each individual device?

johannesjo commented 1 month ago

I have similar issue. Is the Auth Code for Dropbox have to be the same on all synced devices, or generated for each individual device?

Both is possible, but normally it should be the latter.

FlysonBot commented 1 month ago

I really want to provide a procedure to replicate that issue, but it seems that I can easily produce the issue all the time without a procedure. I cannot find a procedure that will ensure 100% replicate the issue on your device so I think it might just be an issue with my specific environment and setup. The only thing I know is, I now backup and created a complete new profile, sync to Dropbox with compression and password enabled, and have this issue. But I don't think compression and password is causing the issue. I also know that when I sync on the android app, if I manually give notification permission, it will run in the background and sync. Even if my wifi shutdown at night and the next day it still sync. But, once I kill the process, there's a good chance it won't sync anymore. On the computer, I guess it is a similar issue. Once the program stop running in the background it can never get back to sync untill I get a new token. It is not an issue with internet connection stopped or token expired, since the app can still sync even after a long period (entire night) of no wifi (thus no communication with Dropbox). And my phone don't have data so it's not connected to anything during the night.

johannesjo commented 1 month ago

Thank you very much for your input. Sounds very annoying. :(

Not sure how to best proceed though, since I am unable to reproduce this. For me dropbox works fine. I created a new dropbox account to see if that makes a difference. So far it hasn't but maybe it takes some time.

johannesjo commented 1 month ago

I am sorry to say, that even with a new account that I cannot reproduce this. :(

FlysonBot commented 1 month ago

Thank you very much for helping out! It's very appreciated how much effort you put into the program, and I think I found something here.

I look at the F12 console on the computer and I noticed that SupProd actually log the access token and expiration time. I realize that the token expired during the night, which is why it wasn't working the next morning. I then think about and I think maybe it just happend that the token expired when I closed my computer and it cannot refresh the token when the computer is closed. So it failed to sync when I open the computer back up again.

The token seems to expire in 4 hours, which seems to be the right at when I close my computer for lunch everyday. So I think it is just a coincidence that the token didn't get refreshed and there is no issue. However, there is an improvement possible at the browser side, which is to refresh the token earlier and not untill the token is about to expire. For example, maybe refresh the token every 30 mins, or every x sync.

On Android's side, there isn't a console so I can't figure out what's going on exactly. Sometimes I close the app the reopen immediately and it failed to sync. Sometimes regardless of how many times I terminate the app it always sync. So that's a mystery that I have not yet figure out why, and I'll keep an eye on it to see if I can spot any pattern. I wonder if token on Android expires faster, or is there a risk of token not being stored before I close the app?

johannesjo commented 1 month ago

Thanks for digging into this. Normally the logic is as follows: Sp makes requests to dropbox as usual with the current access token. If the token is expired for some reason it makes a request with the refreshToken it has saved to get a new accessToken and then retries the previous request. So when the accessToken expires shouldn't matter as much

Maybe tow suggestions:

  1. First update to v 9.0.7 and see if the issue persists.
  2. If this issue persists, it would be great, if you could provide me with the logs from the console when this occurs. This might help me better understand, what is happening. Also a look at the network tab of the dev tools could be helpful, to see if maybe a request fails and for what reasons.
FlysonBot commented 1 month ago

Thank you very much for the response. Both the online and the Android version I used is the newest v9.0.7 release.

On the computer, I believe my previous hypothesis is correct. The updating of token was successfull when the computer is on, but not when the computer goes to sleep. I look at the token expiration timestamp and is expired in 10 minutes, so I closed the laptop and go the lunch. Then I come back and it failed to sync, with the following in the console:

Before I close the computer, when the syncing is still working: Screenshot 2024-08-10 12 57 22

After lunch I come back: image

And the network section that you ask me to take note of: image

(The "已屏蔽:其他" means "Blocked: Other")

And SupProd show a banner saying that Dropbox provided an invalid access token. After I manually authenticate again, I noticed that the access token and refresh token stay the same.

Let me know if you can figure anything out, thanks!

FlysonBot commented 1 month ago

This time I waited exactly at the exact second when the token is going to expire. The 3 seconds before it expire, it sync. The second after it expired, when I try to sync, I got the same thing as above. So it seems that the problem is that the program failed to refresh the token.

fiatphos commented 1 month ago

Hi

I wanted to add my two cents to this issue, hoping it would bring some clarity. The issue is similar, except for the fact that on my macbook (laptop), there is no problem with the syncing (even if I start the app the next day after a long gap). However, on my android phone, if I do not use the app for a while, and then sync again, I get the same 'Initial sync failed' error message.

Additionally, like @FlysonBot, after I enter the authCode for syncing once, and then open the app after some time, I am not able to click on the 'Get Authorization Code' button in the SP app to generate the authCode.

Thank you and appreciate the great work in making this app!

FlysonBot commented 1 month ago

For some reason the botton now works for me. I think I started a complete new profile (backup first) and I haven't have issue with the botton after that. @fiatphos, could you backup and try a new profile and see if the problem got fixed? If so that would be interesting because I don't know how the existed data is supposed to affect the button.

EternityMar22 commented 1 month ago

For some reason the botton now works for me. I think I started a complete new profile (backup first)和I haven't have issue with the botton after that. @fiatphos, could you backup和try a new profile和see if the problem got fixed? If so that would be interesting because I don't know how the existed data is supposed to affect the button.

No, I spent at least 3 hours last night on the sync issue, I tried every possible option and used your suggestion, but in any case, I could only keep my computer's dropbox synced. I can't sync my ios and android phones on the web app (similar to the scenario mentioned upstairs, clear cache can sync the first time, then it will keep reporting an error: sync configuration error or no remote data obtained suggesting an override) I tried webdav sync again, I have my own server but it doesn't work no matter what. I've made sure it's not my server, I even opened the movie on my server with webdav.

FlysonBot commented 1 month ago

From what I know you can't just sync directly with WebDAV becuase of certain safety restrictions (I think is CORP but not sure), so you need to do something before you can sync with WebDAV. But this is from my memory so if something is wrong please point it out, I sometimes got comfused with other stuff. Regarding your issue syncing on IOS and Android, since you mentioned seeing something about "no remote data obtained" and "overide", I supposed you turn on encryption on your computer but you did not enter the same password on your phones. You must enter the password before syncing for it to work, or else it'll suggest you to overide the server data with your local data.

If that is not the case, please describe what you did and what you see. For example, did you get the dropbox authentication dialog showing? Did the authorization botton work for you? Did you get redirected to the dropbox page for authentication? Did you get a token successfully? What happened after you put in the token into the dialog in SupProd? What error did it said?

EternityMar22 commented 1 month ago

From what I know you can't just sync directly with WebDAV becuase of certain safety restrictions (I think is CORP but not sure), so you need to do something before you can sync with WebDAV. But this is from my memory so if something is wrong please point it out, I sometimes got comfused with other stuff. Regarding your issue syncing on IOS and Android, since you mentioned seeing something about "no remote data obtained" and "overide", I supposed you turn on encryption on your computer but you did not enter the same password on your phones. You must enter the password before syncing for it to work, or else it'll suggest you to overide the server data with your local data.

If that is not the case, please describe what you did和what you see. For example, did you get the dropbox authentication dialog showing? Did the authorization botton work for you? Did you get redirected to the dropbox page for authentication? Did you get a token successfully? What happened after you put in the token into the dialog in SupProd? What error did it said?

I never enabled CORS, and regarding the IOS and Android sync issue, I then had encryption not enabled on either of the 2 sides. The whole process of my authorization completed normally and he prompted for a successful sync. But when I close my background, or open my ios chrome after a night. it reports sync failure when loading, reason: failed to get remote data. Suggested I upload locally to overwrite.

The following is the server-side log: 0:28:47+08:00" level=info msg="[INFO] login attempt===> username[xiaoyun] remote_address[13.228.79.13]" time="2024-08-12T10:28:47+08:00" level=info msg="[INFO] user authorized===> username[xiaoyun] remote_address[13.228.79.13] image @FlysonBot @johannesjo HELP ME PLEASE...QAQ

FlysonBot commented 1 month ago

@EternityMar22 It seems that you're using WebDAV. I don't use WebDAV so I have no say over what is happening or how could the problem be fixed. I recommend opening up a new issue (since it is not related to Dropbox anymore) and send a screenshot of what is shown in the console (use F12 for browser version and navigate to console). With more information and opening up a new issue, people can better help you and can focus on 1 problem at a time rather than multiple problems in one issue.

EternityMar22 commented 1 month ago

@EternityMar22 It seems that you're using WebDAV. I don't use WebDAV so I have no say over what is happening or how could the problem be fixed. I recommend opening up a new issue (since it is not related to Dropbox anymore) and send a screenshot of what is shown in the console (use F12 for browser version and navigate to console). With more information and opening up a new issue, people can better help you and can focus on 1 problem at a time rather than multiple problems in one issue.

I manually reproduced the Dropbox synchronization bug again: First, the desktop version of Dropbox on my PC synced everything normally and was always unaffected. Then I used the latest version of the iOS Chrome web app to clear the cache and bind Dropbox. Everything was normal and the synchronization was normal. At this point, I cleared the Chrome background and then reopened the web app. At this point, I was prompted that the initial synchronization failed: the configuration was incorrect, possibly due to a token issue. I then re-obtained the Dropbox authorization code and saved it. The save was successful, but the sync failed, prompting me that I could not retrieve data from the remote: should I upload local data and overwrite it?

The above is the entire process, and end-to-end encryption was not used at any point.

FlysonBot commented 1 month ago

Since it ask you to overwrite, that means the authentication was successful, the app just don't understand the retrieved data. Typically, the only reason this happened is if you encrypted the data so the data is unrecognizable. Since you claimed you had disabled encryption on all devices, I would recommend checking if compression is set to the same (on/off) on all devices. I'm not sure how the compression work but it might affect the data and make it unreadable. If that still doesn't resolve the issue, I recommend deleting the folder in dropbox and sync from your iOS chrome browser before your desktop sync again. Also turn off syncing on desktop and close the app and reopen it after iOS had sync to ensure there was no unintentional bug that causes the encryption to keep happening. If that still produce the problem, only the developer know why. The only cause of this error that I know is data being retrieved but cannot be understand by the program. If you can take a look at F12 and see if there's any error under Console or Network section and put the screenshot here would also help.

EternityMar22 commented 1 month ago

Since it ask you to overwrite, that means the authentication was successful, the app just don't understand the retrieved data. Typically, the only reason this happened is if you encrypted the data so the data is unrecognizable. Since you claimed you had disabled encryption on all devices, I would recommend checking if compression is set to the same (on/off) on all devices. I'm not sure how the compression work but it might affect the data and make it unreadable. If that still doesn't resolve the issue, I recommend deleting the folder in dropbox and sync from your iOS chrome browser before your desktop sync again. Also turn off syncing on desktop and close the app and reopen it after iOS had sync to ensure there was no unintentional bug that causes the encryption to keep happening. If that still produce the problem, only the developer know why. The only cause of this error that I know is data being retrieved but cannot be understand by the program. If you can take a look at F12 and see if there's any error under Console or Network section and put the screenshot here would also help.

First of all, the current situation is that I have to go to dropbox every time to get the authorization code. Because every time I clear the background or enter SP after a long time, it will prompt a configuration error.

I guess this means that the authorization code is invalid.

FlysonBot commented 1 month ago

Yes, if you get a configuration error then your token is probably expired. If you get a request to override data then the app had read the data and cannot understand them, so it ask you to use your local data to override the not-understandable server-side data. The configuration error need to be fixed by the developer, he's trying to figure out what's going on, and it is the issue that I'd been talking about in this issue. Thank you for pointing that out. I also have this syncing issue for a long while, even if I sync single-device.

When sync with Dropbox and you click on the sync botton, on F12 console you should see something like "{Dropbox: {...}, WebDAV: {...}, LocalFile: {...}}". Expanding that and expand Dropbox, you'll see your access token, refresh token, and expiration time of the token. The expiration time is in a timestamp and you can convert it to see when it'll expire. You can see if the syncing issue occured right after the token expiratoin time reached to see if the problem is due to the app failing to refresh the access token (which is my case).

EternityMar22 commented 1 month ago

Yes, if you get a configuration error then your token is probably expired. If you get a request to override data then the app had read the data and cannot understand them, so it ask you to use your local data to override the not-understandable server-side data. The configuration error need to be fixed by the developer, he's trying to figure out what's going on, and it is the issue that I'd been talking about in this issue. Thank you for pointing that out. I also have this syncing issue for a long while, even if I sync single-device.是的,如果您遇到配置错误,那么您的令牌可能已经过期。如果您收到覆盖数据的请求,那么应用程序已经读取了数据,但无法理解这些数据,因此它要求您使用本地数据覆盖无法理解的服务器端数据。配置错误需要由开发人员修复,他正在努力找出问题所在,这也是我在此问题中一直在谈论的问题。感谢您指出这一点。即使我同步单个设备,我也遇到了很长时间的同步问题。

When sync with Dropbox and you click on the sync botton, on F12 console you should see something like "{Dropbox: {...}, WebDAV: {...}, LocalFile: {...}}". Expanding that and expand Dropbox, you'll see your access token, refresh token, and expiration time of the token. The expiration time is in a timestamp and you can convert it to see when it'll expire. You can see if the syncing issue occured right after the token expiratoin time reached to see if the problem is due to the app failing to refresh the access token (which is my case).当与Dropbox同步时,点击同步按钮,在F12控制台上,您应该看到类似"{Dropbox: {...}, WebDAV: {...}, LocalFile: {...}}"的内容。展开该内容并展开Dropbox,您将看到您的访问令牌、刷新令牌和令牌的有效期。有效期以时间戳的形式显示,您可以将其转换为查看其到期时间。您可以查看同步问题是否在令牌到期时间到达后立即出现,以查看问题是否是由于应用程序无法刷新访问令牌(这是我的情况)引起的。

Test results: Assuming that the dropbox configuration file is emptied and the configuration is created by the mobile phone. The result is that both the PC CHROME web version and the PC desktop version can be synchronized with each other, but the iOS Chrome and Safari versions cannot be synchronized, prompting "Cannot read data from the remote, do you want to upload?" The PC version occasionally prompts that the remote data is incomplete. I guess whether it is because the mobile and PC sides use different configuration files? For example, some options are missing.

EternityMar22 commented 1 month ago

I think I found the problem: it seems that the mobile version will report an error when entering "calendar provider error: [object] [object]" and will cause the configuration to be unrecognized. @johannesjo

FlysonBot commented 1 month ago

@EternityMar22 That shouldn't be the problem. The Android app is basically a web app using andorid webview that is put into an app. So the Android app also have CORS (cross-origin-resource-sharing) limitation just like any other thing that uses the browser to access webpage. CORS will prevent SupProd from accessing other website, such as Google Calendar, thus it cannot get the calendar and will throw "calendar provider error: [object] [object]". But that is only to tell you that it cannot get the calendar, it has nothing to do and will not impact syncing. In fact, my Android also have this issue but it does not affect the syncing. I doubt that there are some imcompatibility among different devieces, which the developer should be able to confirm for you (@johannesjo). You can look at the version that you used on different devices and make sure they're the same first.

johannesjo commented 1 month ago

Thank you very much @FlysonBot for helping out! @EternityMar22 could you maybe create a separate issue, since you are using WebDav. It's a bit confusing to me to figure out what exactly is going on here.

About the original issue: @FlysonBot & @fiatphos is there anything I can do to improve the situation? A 504 usually indicates some problem on the server side (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/504. Does the next attempt to get a fresh token work (e.g. after reloading the app)? In this case a simple retry on our side might be good enough.

Also if you click on the request @FlysonBot is there any response body that might help us further?

FlysonBot commented 1 month ago

The 504 issue is never fixed and is always there, unless I reauthenticate the app. If I don't reauthenticate, regardless of how many times I retry I got 504. I found that the app is (1) unable to refresh the access token after it is expired, (2) unable to use an authentication code if the authentication dialog had been closed and reopened, and (3) unable to use the same authentication code twice. (1) will produce a 504, while (2) and (3) will produce a 400.

I alredy refreshed my token so I'll look at the request and response body later. But now from some test I did, I the response behind the 400 to be either invalid request or invalid grant. Invalid request happens when request has an empty code field, such as if I opened the dialog and closed it but the app continuing to make an request with the empty code (which is another issue that should be fixed, since request should not continue if user canceled it). And the invalid grant happens when I use the authentication code in the next dialog, which seems to lead to invalid code verifier. I guess code verifier is a random verifier generated with each dialog so dropbox can tell if this access code is used from the dialog that originall ask for the authentication or from other people.

I'll wait untill the token expired and sent tell you more about the request and response when trying to refresh the token. Also, is there a way to manually trigger process of getting new access token using the refresh token before the token expired, by typing something in the console?

FlysonBot commented 1 month ago

It looks like there is no response body becuase it is 504. But here is what I see when it try to access the token:

Attempt 1 (xhr): General: Request Address: https://api.dropbox.com/oauth2/token Request Method: POST Status Code: 504 Gateway Timeout (From service worker) Referrer Policy: strict-origin-when-cross-origin

Response Header: [None] Request Header: Accept: application/json, text/plain, / Content-Type: application/x-www-form-urlencoded;charset=UTF-8 Referer: https://app.super-productivity.com/ Sec-Ch-Ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126" Sec-Ch-Ua-Mobile: ?0 Sec-Ch-Ua-Platform: "Chrome OS" User-Agent: [my user agent here]

Request Payload: client_id: m7***********ph grant_type: refresh_token refresh_token: [a long token here]

Preview: [nothing seen here] Response: "This request has no response data available."

Attempt 2 (fetch): Request Address: https://api.dropbox.com/oauth2/token Referrer Policy: strict-origin-when-cross-origin [I did not see other things like status code, request method, etc.]

Response Header: [None] Request Header: [exact same as in attempt 1] Request Payload: [exact same as in attempt 1] Preview: "Cannot load preview data: No data found for resource with given identifier." Response: "Cannot load response data: No data found for resource with given identifier."

And here is a screenshot of the console: Screenshot 2024-08-13 18 03 24

johannesjo commented 1 month ago

I guess code verifier is a random verifier generated with each dialog so dropbox can tell if this access code is used from the dialog that originall ask for the authentication or from other people.

Yes you're right, it is. This is why 2 does not work a s you guessed correctly.

Also, is there a way to manually trigger process of getting new access token using the refresh token before the token expired, by typing something in the console?

No unfortunately not.

There is a detail I only realized now. You are using the web version and not the desktop version, right? I find it interesting that the request was triggered from the service worker. I wouldn't expect this to be the case and it might explain why the request fails. You are not using the app for longer than 4 years by any chance? :D

Another thing you might try is stopping the service worker before the refresh happens from the dev tools.

Screenshot from 2024-08-15 19-05-36

FlysonBot commented 1 month ago

Yes I'm using the online version, but remember I have the "initial sync failed" issue on the Android version although not sure if it is the same reason. I tried stopping the service worker but as soon as I click on the sync botton it get started again and the request is again made by the service worker. There's no way to stop it completely, even if I click on a botton that saids unregister. It now even show the service worker is "deleted" but also said it is running at the same time. It seems that as soon as there is a network request, regardless of what the request is, the service worker will be started automatically. Even if I keep clicking on the stop botton non-stop, the request became "pending" and don't actually get a response.

FlysonBot commented 1 month ago

@johannesjo Thank you so much I fixed it! It is indeed the servie worker. After learning from ChatGPT of what it does and look at the interface in more detail, I found a checkbox that said "Bypass for network" (on your screenshot as well), and I checked it, and it synced!!!!!!! So service worker is indeed the issue and we need to bypass "https://api.dropbox.com/oauth2/token" from service worker.

FlysonBot commented 1 month ago

And since the refresh token never expire (or at least won't in the normal use case), I can assume that the syncing issue on Android is also due to not being able to refresh the token, so maybe service worker again. Thank you very much for spending this much time on this issue, and I'm glad that we're able to finally figure out the issue. There had been many times of me thinking that maybe it is my device or some plugin that I used or something, but now we know it is the service worker.

fiatphos commented 1 month ago

So I found a workaround for making the 'Get Authorization Code' button work on my android phone. Previously, I would uninstall the app and reinstall it to be able to tap on the 'Get Authorization Code' button.

However, I discovered that by clearing the storage and the cache on the app settings page (Screenshot) Screenshot_20240816-104236 I was able to get the button enabled again.

Of course, I would do this only after I had synced the data to Dropbox on my laptop and then I would use the 'Keep Remote' option when the app would ask me what to do after I entered the new sync code on the android device.

johannesjo commented 1 month ago

Thank you very much for sharing your insights and spending so much time on this!

Please let me know if this ever reoccurs!

Might be that this was just an outdated service worker from a previous version, but maybe there is also something not configured correctly, although to me it doesn't look that there is anything explaining this behavior: https://github.com/johannesjo/super-productivity/blob/master/ngsw-config.json

FlysonBot commented 1 month ago

This is always happening on my machine so unless I "Bypass to network", otherwise I'll keep encountering this issue using the browser version. Since it is a service worker issue, I guess you could replicate it on your browser and figure out what's wrong? It keeps happening on my side and I don't know if there're side-effect to bypassing service worker, such as slow resource request, so I only bypass it when I need to sync. The syncing issue still does occur on the Android app once the app is no longer running in background (especially when low battery it got killed all the time). I don't know if anyone else is experiencing that and what is causing it, since there isn't a development tool for the android app that I could invest some time to take a look at.

johannesjo commented 1 month ago

Thanks for the clarification! It's interesting to me, why I was unable to reproduce this from my browser or from my android version, but I will investigate more once I am done with all the changes for the next pre release.

FlysonBot commented 1 month ago

I open the website in incognito and no longer experience the problem, so I believe it might be some of my extentions or settings. I'm trying to figure it out. I think the extention Privacy Badger from EFF might be causing the issue, although I told it to disable on this website. I turn off my extentions and refresh and untill I disabled Privacy Badger it was able to sync after an refresh, but I still need more reproducing and verifying to make sure it is causing the issue and it is the only thing causing the issue.

While my test I noticed something interesting which is when I open multiple instances, the second instance experienced "initial sync failed" error while the first instance still work. After I closed the second instance and refresh the first, the first experience "initial sync failed" as well. I cannot tell if (1) opening a second instance is causing the problem, or (2) there was already a problem but it will only occur once I reload the website (such as refreshing or opening a second instance). If (2) is true, then I guess the issue on the Android app could be similar, something goes wrong already while the app is running but only occur after the app is terminated somehow thus the syncing will work untill app restarted. Because I also get the "initial sync failed" error on Android app.

FlysonBot commented 1 month ago
  1. After adding Privacy Badger to the incognito I start encountering issue, and the issue disappeared once it is diabled. So I can confirm it was the Privacy Badger extention that is causing the failure to refresh access token and is the only cause of the issue, on my browser.

  2. I'm also able to confirm that opening up a second instance doens't lead to "initial sync failed" issue, which means something went wrong in the background but the currently running instance will not be affected.

  3. When "Initial Sync Failed" error occured, in the console the dropbox data became one of the followings (no parameters means did not see those keys in the object):

Regardless of what is cuasing this issue to occur, it seems that "initial sync failed" (or "Sync authentication failed, please check your configuration!" if you retry a second time) occures when the access token is not being retrieved from app's data. That is a separate issue and is likely the issue on the Android, since I always see "initial sync failed" on Android. @johannesjo Do you think the token is sometimes not properly stored or retrieved by the app?

FlysonBot commented 1 month ago

Something important is that when the "initial sync failed" issue occured, if there are other running instances, the other running instances will keep syncing without problems. However, their access token also became null and refresh token disappeared as in case 1. I don't know how they're supposed to keep syncing that way but they still do sync, I tested it. As said earlier, this issue is not caused by running multiple instances together, I'm just describing what I'll see if I happen to run multiple instances when issue occured. It is also unrelated to how many client is syncing with the Dropbox, so it has to be an app issue itself. I also have this issue on Android app and in incognito mode so I think this time it's not extentions.

johannesjo commented 3 weeks ago

Do you think the token is sometimes not properly stored or retrieved by the app?

I wouldn't rule it out, but I don't think it is very likely either.

fiatphos commented 3 weeks ago

I received an update for the app on the Google Play Store three days ago, and after the update, the sync seems to be working well with Dropbox. Also, now when I use the 'Get Authorization Code' button, I receive a menu which asks whether I want to 'Leave this page' or 'Stay on the same page'. Tapping on 'Leave this page' opens up the browser window where I can access the auth code, thereby solving the problem of the button not working as well.