search

xbrowsersync / app

xBrowserSync browser extensions / mobile app
GNU General Public License v3.0
1.52k stars 101 forks source link

Multiple changes trigger full reset and resync #222

Open viharm opened 4 years ago

viharm commented 4 years ago

Issue description

Whenever I make many (sometimes even as low as two) changes to my bookmarks, the client triggers a full reset and resync of all bookmarks from the server. Am I doing something wrong? Is there a configuration setting I can change to fix this?

Environment

API (server)

API service version: 1.1.10 (self hosted docker image). MongoDB version: 4.2.0 (docker image). Network: Bridge. OpenMediaVault Docker GUI plugin version: 4.1.6. Docker CE server: Engine 19.03.11; API 1.40; Go 1.13.10; Containerd 1.2.13; runc 1.0.0-rc10; docker-init 0.18.0. Docker CE client: Engine 19.03.11; API 1.40; Go 1.13.10. Docker Compose version: 1.8.0-2. OS: OpenMediaVault 4.1.35-1 (Arrakis; based on Debian GNU/Linux 9.12 Stretch). Linux kernel: 4.19.0-0.bpo.8-amd64 xbrowsersync/api#1 SMP Debian 4.19.98-1~bpo9+1 (2020-03-09) x86_64 GNU/Linux. Filesystem: ext4.

Client

Client add-on version: 1.5.2. Browser: Firefox x86-64, 78.0.1. Operating System: Kubuntu 20.04. KDE Plasma Version: 5.18.5. KDE Frameworks Version: 5.68.0. Qt Version: 5.12.8. Kernel Version: 5.4.0-40-generic. OS Type: 64-bit. Processors: 4× Intel® Celeron® CPU N3150 @ 1.60GHz.

nero120 commented 4 years ago

This is likely an extension issue not API, can you post the log from the extension please?

viharm commented 4 years ago

This is likely an extension issue not API, can you post the log from the extension please?

I'm afraid I don't know how to get the logs from extensions. Can you please point me in the right direction?

nero120 commented 4 years ago

I'm afraid I don't know how to get the logs from extensions. Can you please point me in the right direction?

Open the xBrowserSync extension window, click on the cog icon in the top right to get to the Settings panel, then scroll down to Issues and click the Download Log button.

protocolist commented 4 years ago

I have this same issue and i'm happy to assist with troubleshooting if you need.

nero120 commented 4 years ago

@protocolist can you post your log please?

protocolist commented 4 years ago

Sure. I cleared the log and then attempted an operation where I moved three bookmarks from one folder to another. This immediately triggered the reset and sync, and the three bookmarks remained in the original location. Browser is Edge 84.0.522.40, I have also experienced the problem on ungoogled-chromium 81.0.4044.138.

Let me know if you need more detail.

xbs_log_20200731002309.txt

nero120 commented 4 years ago

@protocolist yes the log confirms it was a multiple move that triggered the re-sync. did you move the bookmarks to a folder that was a sibling of the bookmarks?

protocolist commented 4 years ago

That's correct. I've just done some experimentation moving the same three bookmarks around:

Moved down to a nested folder from its parent folder - triggers reset Moved up from a nested folder to its parent folder - triggers reset Moved up from a nested folder to a different folder at the same hierarchical level as the parent folder - okay Moved up from a nested folder to the root folder - okay

I then reduced the number of bookmarks I was attempting to move to two, same as above

Finally, I reduced the number to one, same problem.

As such, I think that it's possible i'm reporting a different bug to the original poster, or alternatively, this issue is typically only noticed while moving multiple bookmarks.

Log from a single move is attached. I did a succesful move of the same object just before in this same log. xbs_log_20200731082101.txt

viharm commented 4 years ago

@protocolist , unfortunately I haven't been able to test it systematically as you have. However the symptoms are very similar. Are you using your own API server, or an existing public service?

@nero120 , unfortunately, I had to upgrade my OMV installation, so couldn't get to my log as requested. I will do so as soon as I can. However I can confirm that the symptoms are very similar to those described by @protocolist .

protocolist commented 4 years ago

I'm on the official API, https://api.xbrowsersync.org/

nero120 commented 4 years ago

@viharm, this issue is not linked to the API, it's a bug in the app. This issue is in the wrong repo so I will move it to app. Can you please do the following test to confirm:

  1. Whilst synced, open the native browser bookmark manager.
  2. In any folder, create two bookmarks and one folder.
  3. Select both bookmarks and drag them into the sibling folder.

@protocolist your issue is actually already logged here: #158. I've already fixed it in dev and will be included in the upcoming release (v1.6.0).

protocolist commented 4 years ago

@nero120 Many thanks

viharm commented 4 years ago

@nero120 , I followed the three steps you mentioned in your comment on 31-Jul-2020. Here's the excerpt from the log which aligns with the time of carrying out the steps.

2020-10-05 22:48:44.959 [trace] onCreated event detected
2020-10-05 22:48:45.964 [trace] Sync zthwmo (push) queued
2020-10-05 22:48:46.256 [trace] Processing sync zthwmo (0 in queue)
2020-10-05 22:48:46.368 [trace] Remote bookmarks data updated at 2020-10-05T22:48:46.374Z
2020-10-05 22:48:55.830 [trace] onChanged event detected
2020-10-05 22:48:56.838 [trace] Sync kbqgiv (push) queued
2020-10-05 22:48:57.139 [trace] Processing sync kbqgiv (0 in queue)
2020-10-05 22:48:57.230 [trace] Remote bookmarks data updated at 2020-10-05T22:48:57.236Z
2020-10-05 22:49:00.263 [trace] onChanged event detected
2020-10-05 22:49:01.268 [trace] Sync 5kqbim (push) queued
2020-10-05 22:49:01.585 [trace] Processing sync 5kqbim (0 in queue)
2020-10-05 22:49:01.683 [trace] Remote bookmarks data updated at 2020-10-05T22:49:01.669Z
2020-10-05 22:49:04.044 [trace] onCreated event detected
2020-10-05 22:49:05.048 [trace] Sync nsjr4w (push) queued
2020-10-05 22:49:05.340 [trace] Processing sync nsjr4w (0 in queue)
2020-10-05 22:49:05.413 [trace] Remote bookmarks data updated at 2020-10-05T22:49:05.419Z
2020-10-05 22:49:09.468 [trace] onChanged event detected
2020-10-05 22:49:10.472 [trace] Sync 122ivl6 (push) queued
2020-10-05 22:49:10.767 [trace] Processing sync 122ivl6 (0 in queue)
2020-10-05 22:49:10.847 [trace] Remote bookmarks data updated at 2020-10-05T22:49:10.853Z
2020-10-05 22:49:10.944 [trace] onChanged event detected
2020-10-05 22:49:11.950 [trace] Sync 3tucgt (push) queued
2020-10-05 22:49:12.228 [trace] Processing sync 3tucgt (0 in queue)
2020-10-05 22:49:12.298 [trace] Remote bookmarks data updated at 2020-10-05T22:49:12.308Z
2020-10-05 22:49:14.337 [trace] onChanged event detected
2020-10-05 22:49:15.342 [trace] Sync 8d6wtt (push) queued
2020-10-05 22:49:15.605 [trace] Processing sync 8d6wtt (0 in queue)
2020-10-05 22:49:15.678 [trace] Remote bookmarks data updated at 2020-10-05T22:49:15.684Z
2020-10-05 22:49:19.187 [trace] onCreated event detected
2020-10-05 22:49:19.197 [trace] onMoved event detected
2020-10-05 22:49:20.203 [trace] Sync q8dqng (push) queued
2020-10-05 22:49:20.206 [trace] Sync 1mdzay5 (push) queued
2020-10-05 22:49:20.822 [trace] Processing sync q8dqng (1 in queue)
2020-10-05 22:49:20.844 [trace] Processing sync 1mdzay5 (0 in queue)
2020-10-05 22:49:20.914 [trace] Remote bookmarks data updated at 2020-10-05T22:49:20.921Z
2020-10-05 22:49:22.825 [trace] onChanged event detected
2020-10-05 22:49:23.832 [trace] Sync lcnrjp (push) queued
2020-10-05 22:49:24.105 [trace] Processing sync lcnrjp (0 in queue)
2020-10-05 22:49:24.191 [trace] Remote bookmarks data updated at 2020-10-05T22:49:24.202Z
2020-10-05 22:49:30.579 [trace] onMoved event detected
2020-10-05 22:49:30.585 [trace] onMoved event detected
2020-10-05 22:49:31.588 [trace] Sync sfp121 (push) queued
2020-10-05 22:49:31.591 [trace] Sync 1j3jjck (push) queued
2020-10-05 22:49:32.346 [trace] Processing sync sfp121 (1 in queue)
2020-10-05 22:49:32.371 [trace] Processing sync 1j3jjck (0 in queue)
2020-10-05 22:49:32.443 [trace] Remote bookmarks data updated at 2020-10-05T22:49:32.450Z
2020-10-05 22:50:52.781 [trace] Downloading file xbs_log_20201005235052.txt

There was no unusual activity detected. OS: Windows 10 Home 1909 build 18363.1082 64-bit Browser: Firefox 80.0.1 Browser extension: 1.5.2 API: Self hosted v1.1.12.

I will try again with a larger change set and let you know.

nero120 commented 4 years ago

@viharm looks like the move was successfully synced so not the issue I suspected it might be. You could try with the latest build as it's possible your original issue has already been resolved?

viharm commented 4 years ago

@nero120 , yes it seemed to have worked when I tried. I will try a couple of more times and will try recreate the issue.

nero120 commented 4 years ago

Great stuff @viharm, fingers crossed it's already sorted.

viharm commented 4 years ago

@nero120 , I tried again, the steps you suggested. This time it triggered a full reset, and restored the state right before the final move (creation of the three bookmarks and the folders was successfully synced; it failed when I moved them into the sibling folder).

Here's the log

2020-10-17 18:43:35.319 [trace] onCreated event detected
2020-10-17 18:43:36.350 [trace] Sync h721nk (push) queued
2020-10-17 18:43:36.899 [trace] Processing sync h721nk (0 in queue)
2020-10-17 18:43:37.041 [trace] Remote bookmarks data updated at 2020-10-17T18:43:37.069Z
2020-10-17 18:43:37.399 [trace] onRemoved event detected
2020-10-17 18:43:38.423 [trace] Sync 14vyxyq (push) queued
2020-10-17 18:43:39.016 [trace] Processing sync 14vyxyq (0 in queue)
2020-10-17 18:43:39.108 [trace] Remote bookmarks data updated at 2020-10-17T18:43:39.134Z
2020-10-17 18:43:51.506 [trace] onCreated event detected
2020-10-17 18:43:52.510 [trace] Sync 5zkanj (push) queued
2020-10-17 18:43:53.243 [trace] Processing sync 5zkanj (0 in queue)
2020-10-17 18:43:53.353 [trace] Remote bookmarks data updated at 2020-10-17T18:43:53.376Z
2020-10-17 18:43:55.487 [trace] onMoved event detected
2020-10-17 18:43:56.499 [trace] Sync al5dkd (push) queued
2020-10-17 18:43:57.339 [trace] Processing sync al5dkd (0 in queue)
2020-10-17 18:43:57.425 [trace] Remote bookmarks data updated at 2020-10-17T18:43:57.451Z
2020-10-17 18:44:03.363 [trace] onCreated event detected
2020-10-17 18:44:04.382 [trace] Sync xchq96 (push) queued
2020-10-17 18:44:04.828 [trace] Processing sync xchq96 (0 in queue)
2020-10-17 18:44:04.913 [trace] Remote bookmarks data updated at 2020-10-17T18:44:04.938Z
2020-10-17 18:44:05.260 [trace] onMoved event detected
2020-10-17 18:44:06.282 [trace] Sync 1ipc3uz (push) queued
2020-10-17 18:44:06.840 [trace] Processing sync 1ipc3uz (0 in queue)
2020-10-17 18:44:06.984 [trace] Remote bookmarks data updated at 2020-10-17T18:44:07.007Z
2020-10-17 18:44:08.511 [trace] onCreated event detected
2020-10-17 18:44:09.521 [trace] Sync me2ex0 (push) queued
2020-10-17 18:44:09.969 [trace] Processing sync me2ex0 (0 in queue)
2020-10-17 18:44:10.050 [trace] Remote bookmarks data updated at 2020-10-17T18:44:10.075Z
2020-10-17 18:44:10.788 [trace] onMoved event detected
2020-10-17 18:44:11.795 [trace] Sync 19gj352 (push) queued
2020-10-17 18:44:12.390 [trace] Processing sync 19gj352 (0 in queue)
2020-10-17 18:44:12.473 [trace] Remote bookmarks data updated at 2020-10-17T18:44:12.498Z
2020-10-17 18:44:16.472 [trace] onCreated event detected
2020-10-17 18:44:17.480 [trace] Sync u5t8qy (push) queued
2020-10-17 18:44:17.982 [trace] Processing sync u5t8qy (0 in queue)
2020-10-17 18:44:18.054 [trace] Remote bookmarks data updated at 2020-10-17T18:44:18.081Z
2020-10-17 18:44:18.332 [trace] onRemoved event detected
2020-10-17 18:44:19.340 [trace] Sync 1b5exgf (push) queued
2020-10-17 18:44:19.769 [trace] Processing sync 1b5exgf (0 in queue)
2020-10-17 18:44:19.839 [trace] Remote bookmarks data updated at 2020-10-17T18:44:19.866Z
2020-10-17 18:44:20.392 [trace] onCreated event detected
2020-10-17 18:44:20.434 [trace] onMoved event detected
2020-10-17 18:44:21.441 [trace] Sync c6gid6 (push) queued
2020-10-17 18:44:21.511 [trace] Sync 9xeowz (push) queued
2020-10-17 18:44:22.558 [trace] Processing sync c6gid6 (1 in queue)
2020-10-17 18:44:22.638 [trace] Processing sync 9xeowz (0 in queue)
2020-10-17 18:44:22.762 [trace] Remote bookmarks data updated at 2020-10-17T18:44:22.787Z
2020-10-17 18:44:32.721 [trace] onChanged event detected
2020-10-17 18:44:33.731 [trace] Sync 12e3kpf (push) queued
2020-10-17 18:44:34.247 [trace] Processing sync 12e3kpf (0 in queue)
2020-10-17 18:44:34.330 [trace] Remote bookmarks data updated at 2020-10-17T18:44:34.355Z
2020-10-17 18:44:39.238 [trace] onMoved event detected
2020-10-17 18:44:40.248 [trace] Sync 19nydbu (push) queued
2020-10-17 18:44:41.088 [trace] Processing sync 19nydbu (0 in queue)
2020-10-17 18:44:41.183 [trace] Remote bookmarks data updated at 2020-10-17T18:44:41.209Z
2020-10-17 18:44:42.291 [trace] onMoved event detected
2020-10-17 18:44:42.321 [trace] onMoved event detected
2020-10-17 18:44:42.340 [trace] onMoved event detected
2020-10-17 18:44:43.326 [trace] Sync 1669nrl (push) queued
2020-10-17 18:44:43.379 [trace] Sync 1afxopg (push) queued
2020-10-17 18:44:43.397 [trace] Sync 1gp069w (push) queued
2020-10-17 18:44:45.696 [trace] Processing sync 1669nrl (2 in queue)
2020-10-17 18:44:45.733 [trace] Sync 1669nrl failed
2020-10-17 18:44:45.757 [error] bookmarks.sync: [10111] XBookmarkNotFound   addNewInXBookmarks@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:2563:13 bookmarksMoved/<@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:3864:65 k/<@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:140:44 $digest@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:151:185 $evalAsync/<@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:154:326 Tg/this.completeTask@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:166:15 Ag/h.defer/d<@moz-extension://d588932e-8334-4190-a5ad-ec18ea0860de/js/background.min.js:52:3 
2020-10-17 18:44:45.785 [trace] {"bookmark":{"id":"AwjabUdwNSqY","title":"Welcome to AMD ׀ High-Performance Processors and Graphics","index":0,"dateAdded":1602960231494,"type":"bookmark","url":"https://www.amd.com/en","parentId":"WyqGDt0x8CTm"},"container":"[xbs] Menu","indexPath":[18,4],"targetInfo":{"bookmark":{"id":"AwjabUdwNSqY","title":"Welcome to AMD ׀ High-Performance Processors and Graphics","index":0,"dateAdded":1602960231494,"type":"bookmark","url":"https://www.amd.com/en","parentId":"WyqGDt0x8CTm"},"container":"[xbs] Menu","indexPath":[18,4,0],"syncChange":true},"type":4,"syncChange":true}
2020-10-17 18:44:45.805 [trace] Sync 3d48i5 (pull) queued
2020-10-17 18:44:45.823 [trace] Processing sync 3d48i5 (0 in queue)
2020-10-17 18:44:56.050 [trace] Local population completed in 8.79s
2020-10-17 18:44:56.081 [trace] Local sync data refreshed
nero120 commented 4 years ago

@viharm I cannot replicate this in Firefox v81.0.2 on the latest build. If I move multiple bookmarks into sibling folders it syncs fine. Can you please provide me with steps to replicate the error?

viharm commented 4 years ago

Environment

API (server)

API service version: 1.1.12 (self hosted docker image). MongoDB version: 4.4.1. Network: Host. Docker CE server: Engine 19.03.13~3-0~debian-buster; API 1.40; Go 1.13.15; Containerd 1.3.7; runc 1.0.0-rc10; docker-init 0.18.0. Docker CE client: Engine 19.03.13; API 1.40; Go 1.13.15. Docker Compose version: 1.21.0-3. OS: OpenMediaVault 5.5.10-1 (Usul; based on Debian GNU/Linux 10 Buster). Linux kernel: 5.7.0-0.bpo.2-amd64 #1 SMP Debian 5.7.10-1~bpo10+1 (2020-07-30) x86_64 GNU/Linux. Filesystem: ext4.

Client

Client add-on version: 1.5.2. Browser: Firefox x86-64, 81.0.2. Operating System: Kubuntu 20.04. KDE Plasma Version: 5.18.5. KDE Frameworks Version: 5.68.0. Qt Version: 5.12.8. Kernel Version: 5.4.0-40-generic. OS Type: 64-bit. Processors: 4× Intel® Core™ i5-6300U CPU @ 2.40GHz.

Steps taken

  1. Go to amd.com
  2. Add to bookmarks.
  3. Select a parent folder (probably triggers a "move" event)
  4. Go to intel.com
  5. Add to bookmarks.
  6. Select a parent folder (probably triggers a "move" event)
  7. Go to qualcomm.com
  8. Add to bookmarks.
  9. Select a parent folder (probably triggers a "move" event)
  10. Open bookmarks in the Firefox library.
  11. Add a new folder XbsTest alongside the three new bookmarks in the same parent folder
  12. [All OK and synced so far]
  13. Move the three new bookmarks to the XbsTest folder.
  14. Reset triggered.
viharm commented 4 years ago

OK, it seems like I may not have the latest build of the client app (browser extension). Would you recommend trying the pre-release? Should I just install from the zip download?

nero120 commented 4 years ago

Yes use the latest build, download xbrowsersync_1.6.0.994_firefox.zip and install this (ideally using a new firefox profile so you don't affect your actual data) as a temporary extension. Just browse to:

about:debugging#/runtime/this-firefox

then click on "Load temporary add-on" and select the zip file. This will allow you to test the extension using a new sync id without affecting your real profile data, and the temporary extension will be automatically removed once you shut down firefox.

jason076 commented 3 years ago

I have the same issue, but in my case it is enough to make one change, to trigger a complete resync. If I add something to the bookmark bar in my chromium and open my firefox afterwards, the firefox does a complete resync. 

2021-04-02 08:26:16.817 [trace] Starting up
2021-04-02 08:26:16.941 [trace] {"checkForAppUpdates":true,"displayHelp":false,"displayOtherSyncsWarning":false,"lastUpdated":"2021-04-01T17:33:22.754Z","syncBookmarksToolbar":true,"syncEnabled":true,"syncId":"f8c35db5128e4db3bd69c68fcb9a604a","syncVersion":"1.5.2","appVersion":"1.5.2","platform":{"name":"firefox","version":"87.0.0","mobile":false,"os":"Linux x86_64"}}
2021-04-02 08:26:19.343 [trace] Updates available, local:2021-04-01T17:33:22.754Z remote:2021-04-02T08:26:01.945Z
2021-04-02 08:26:19.347 [trace] Sync 1gihrly (pull) queued
2021-04-02 08:26:19.350 [trace] Processing sync 1gihrly (0 in queue)
2021-04-02 08:26:27.600 [trace] Waiting for syncs to finish...
2021-04-02 08:26:32.065 [trace] Local population completed in 11.004s
2021-04-02 08:26:32.645 [trace] Syncs finished, resuming

Both browsers use extension version 1.5.2

nero120 commented 3 years ago

@jason076 that's by design. When you opened firefox it detected sync updates and refreshed your local bookmarks to match the synced data.

jason076 commented 3 years ago

@nero120 That does not inspire much confidence into the sync service, if I open my browser and see all bookmarks disappearing. It gives me the feeling that the sync is broken, and I maybe loose some bookmarks.

nero120 commented 3 years ago

Don't use it then.

jason076 commented 3 years ago

@nero120 Since you told me, that this is by design, I tested to create a folder in firefox and then open my chromium to verify if the resync happens there to, because I did not recognize a resync in chromium before.

I found the reason, why the experience in firefox feels broken. The resync is very slow. I see every single folder disappearing and then reappearing after a while. The full resync takes more than 10 seconds. In chromium this resync is barely noticeable since it takes less than 1s. I created an issue for that #291

nero120 commented 3 years ago

@jason076 that's a known issue caused by Firefox's web extension API implementation. There is already a feature request for your original issue where bookmarks are deleted and then recreated, if you want to add your support for that feature do so there.