logseq / logseq

A privacy-first, open-source platform for knowledge management and collaboration. Download link: http://github.com/logseq/logseq/releases. roadmap: http://trello.com/b/8txSM12G/roadmap
https://logseq.com
GNU Affero General Public License v3.0
32.73k stars 1.92k forks source link

Syncing was going in cycle and never ends #11086

Closed stucash closed 1 week ago

stucash commented 8 months ago

Search first

What Happened?

Yesterday after I’ve updated some of my notes I found the syncing stopped working abruptly (part of my update went through), the icon was kept in orange status and when drilled into details I could see that my files kept being processed. If I delete this file, then Logseq would pick another file, which was kept in processing status.

Reproduce the Bug

Log off Log on Turn on Sync and smart syncing Click on Sync icon and type in password Syncing starts.

Expected Behavior

I expected the syncing to finish soon as it was just a small batch of files.

Screenshots

No response

Desktop or Mobile Platform Information

System:    Kernel: 5.18.0-4mx-amd64 [5.18.16-1~mx21+1] x86_64 bits: 64 compiler: gcc v: 10.2.1 
           parameters: BOOT_IMAGE=/antiX/vmlinuz quiet splasht nosplash lang=en_GB kbd=gb 
           tz=Asia/Shanghai persist_static from=usb savestate splasht xorg=nvidia 
           Desktop: Xfce 4.18.1 tk: Gtk 3.24.24 info: xfce4-panel wm: xfwm 4.18.0 vt: 7 
           dm: LightDM 1.26.0 Distro: MX-21.3_ahs_x64 Wildflower September 18  2022 
           base: Debian GNU/Linux 11 (bullseye) 
Machine:   Type: Desktop Mobo: Gigabyte model: Z97X-Gaming 5 v: x.x serial: <filter> 
           UEFI: American Megatrends v: F7 date: 09/18/2015 
CPU:       Info: Quad Core model: Intel Core i7-4790K bits: 64 type: MT MCP arch: Haswell 
           family: 6 model-id: 3C (60) stepping: 3 microcode: 1C cache: L2: 8 MiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 64004 
           Speed: 1083 MHz min/max: 800/4600 MHz Core speeds (MHz): 1: 1083 2: 2896 3: 1285 
           4: 1146 5: 1477 6: 1135 7: 1291 8: 1188 
           Vulnerabilities: Type: itlb_multihit status: KVM: VMX disabled 
           Type: l1tf mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable 
           Type: mds status: Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable 
           Type: meltdown mitigation: PTI 
           Type: mmio_stale_data status: Not affected 
           Type: retbleed status: Not affected 
           Type: spec_store_bypass status: Vulnerable 
           Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization 
           Type: spectre_v2 
           mitigation: Retpolines, STIBP: disabled, RSB filling, PBRSB-eIBRS: Not affected 
           Type: srbds status: Vulnerable: No microcode 
           Type: tsx_async_abort status: Not affected 
Graphics:  Device-1: Intel Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics 
           vendor: Gigabyte driver: i915 v: kernel bus-ID: 00:02.0 chip-ID: 8086:0412 
           class-ID: 0300 
           Device-2: NVIDIA GM206 [GeForce GTX 960] vendor: Gigabyte driver: nvidia v: 525.125.06 
           bus-ID: 01:00.0 chip-ID: 10de:1401 class-ID: 0300 
           Display: x11 server: X.Org 1.20.14 compositor: xfwm4 v: 4.18.0 driver: loaded: nvidia 
           display-ID: :0.0 screens: 1 
           Screen-1: 0 s-res: 4080x2160 s-dpi: 96 s-size: 1080x572mm (42.5x22.5") 
           s-diag: 1222mm (48.1") 
           Monitor-1: HDMI-0 res: 1920x1080 hz: 60 dpi: 93 size: 527x296mm (20.7x11.7") 
           diag: 604mm (23.8") 
           Monitor-2: DP-0 res: 1920x1080 hz: 60 dpi: 93 size: 527x296mm (20.7x11.7") 
           diag: 604mm (23.8") 
           Monitor-3: DP-2 res: 1080x1920 hz: 60 
           Monitor-4: DP-4 res: 1080x1920 hz: 60 
           OpenGL: renderer: NVIDIA GeForce GTX 960/PCIe/SSE2 v: 4.6.0 NVIDIA 525.125.06 
           direct render: Yes 
Audio:     Device-1: Intel Xeon E3-1200 v3/4th Gen Core Processor HD Audio driver: snd_hda_intel 
           v: kernel bus-ID: 00:03.0 chip-ID: 8086:0c0c class-ID: 0403 
           Device-2: Intel 9 Series Family HD Audio vendor: Gigabyte driver: snd_hda_intel 
           v: kernel bus-ID: 00:1b.0 chip-ID: 8086:8ca0 class-ID: 0403 
           Device-3: NVIDIA GM206 High Definition Audio vendor: Gigabyte driver: snd_hda_intel 
           v: kernel bus-ID: 01:00.1 chip-ID: 10de:0fba class-ID: 0403 
           Sound Server-1: ALSA v: k5.18.0-4mx-amd64 running: yes 
           Sound Server-2: PulseAudio v: 14.2 running: yes 
           Sound Server-3: PipeWire v: 0.3.19 running: no 
Network:   Device-1: Qualcomm Atheros Killer E220x Gigabit Ethernet vendor: Gigabyte driver: alx 
           v: kernel port: d000 bus-ID: 03:00.0 chip-ID: 1969:e091 class-ID: 0200 
           IF: eth0 state: down mac: <filter> 
           Device-2: Intel Wireless-AC 9260 driver: iwlwifi v: kernel modules: wl port: d000 
           bus-ID: 06:00.0 chip-ID: 8086:2526 class-ID: 0280 
           IF: wlan0 state: up mac: <filter> 
Bluetooth: Device-1: Cambridge Silicon Radio Bluetooth Dongle (HCI mode) type: USB driver: btusb 
           v: 0.8 bus-ID: 3-4:2 chip-ID: 0a12:0001 class-ID: e001 
           Report: hciconfig ID: hci0 rfk-id: 1 state: up address: <filter> bt-v: 2.1 lmp-v: 4.0 
           sub-v: 22bb hci-v: 4.0 rev: 22bb 
           Info: acl-mtu: 310:10 sco-mtu: 64:8 link-policy: rswitch hold sniff park 
           link-mode: slave accept service-classes: rendering, capturing, object transfer, audio 
           Device-2: Intel Wireless-AC 9260 Bluetooth Adapter type: USB driver: btusb v: 0.8 
           bus-ID: 3-8:4 chip-ID: 8087:0025 class-ID: e001 
           Report: ID: hci1 rfk-id: 2 state: up address: <filter> bt-v: 3.0 lmp-v: 5.1 sub-v: 100 
           hci-v: 5.1 rev: 100 
           Info: acl-mtu: 1021:4 sco-mtu: 96:6 link-policy: rswitch sniff link-mode: slave accept 
           service-classes: rendering, capturing, object transfer, audio 
Drives:    Local Storage: total: 1.93 TiB used: 684.11 GiB (34.6%) 
           SMART Message: Unable to run smartctl. Root privileges required. 
           ID-1: /dev/sda maj-min: 8:0 vendor: Samsung model: SSD 840 EVO 120GB size: 111.79 GiB 
           block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s type: SSD serial: <filter> 
           rev: BB6Q scheme: MBR 
           ID-2: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST31000524AS size: 931.51 GiB 
           block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s type: HDD rpm: 7200 
           serial: <filter> rev: JC4B scheme: MBR 
           ID-3: /dev/sdc maj-min: 8:32 type: USB vendor: Seagate 
           model: BarraCuda 120 SSD ZA1000CM10003 size: 931.51 GiB block-size: physical: 512 B 
           logical: 512 B type: SSD serial: <filter> scheme: GPT 
Swap:      Kernel: swappiness: 15 (default 60) cache-pressure: 100 (default) 
           ID-1: swap-1 type: file size: 10 GiB used: 0 KiB (0.0%) priority: -2 
           file: /live/boot-dev/swap-file 
Sensors:   System Temperatures: cpu: 29.8 C mobo: 27.8 C gpu: nvidia temp: 34 C 
           Fan Speeds (RPM): N/A gpu: nvidia fan: 22% 
Info:      Processes: 325 Uptime: 37m wakeups: 1 Memory: 19.41 GiB used: 4.79 GiB (24.7%) 
           Init: SysVinit v: 2.96 runlevel: 5 default: 5 tool: systemctl Compilers: gcc: 10.2.1 
           alt: 10 clang: 11.0.1-2 Client: shell wrapper v: 5.1.4-release inxi: 3.3.06 
Boot Mode: UEFI
Video Tweaks:
Detected possible Hybrid Graphics
No kernel video driver

Additional Context

Error Log from dev console:

instrument.js:191 [SyncManager] need-password -> idle :reason nil :local-txid 252 :args nil :now 2024-03-05T08:02:57.447Z
instrument.js:191 [SyncManager] idle -> remote->local-full-sync :reason nil :local-txid 252 :args nil :now 2024-03-05T08:02:57.448Z
instrument.js:191 [01b1f34f-8489-4276-9d78-95b5af6a68db] setting sync age-encryption passphrase...
instrument.js:191 [SyncManager] remote->local-full-sync -> idle :reason nil :local-txid 252 :args nil :now 2024-03-05T08:02:57.528Z
instrument.js:191 [SyncManager] idle -> local->remote-full-sync :reason nil :local-txid 252 :args nil :now 2024-03-05T08:02:57.529Z
instrument.js:191 :frontend.fs.sync/diff-file-metadata-sets-elapsed-time 5.900000000139698 ms
instrument.js:191 [full-sync(local->remote)] 12 files need to sync and 0 local files need to delete
instrument.js:191 [full-sync(remote->local)] 1 files need to sync
instrument.js:191 :sync-local->remote change [pages/读书___疑问___马拉松训练指南.md assets/训练量表格3.jpeg pages/读书___笔记___马拉松训练指南.md assets/training_cycle_two.jpeg assets/training_cycle_three.jpeg journals/2024_03_04.md assets/训练量表格2.jpeg assets/training_cycle_one.jpeg assets/训练量表格1.jpeg assets/training_cycle_four.jpeg assets/vertical_oscillation.mp4 assets/Electric-Blues-DanielsTables-3-01-02.xlsm]
instrument.js:191 "[diff-merge]base!=current, 3-way merge"
instrument.js:191 Parsing start:  pages/读书___笔记___马拉松训练指南.md
instrument.js:191 Parsing finished:  pages/读书___笔记___马拉松训练指南.md
instrument.js:191 [SyncManager] local->remote-full-sync -> idle :reason nil :local-txid 252 :args nil :now 2024-03-05T08:02:58.856Z
instrument.js:191 rsapi error: Error: Error invoking remote method 'main': Error: Can not upload temp file, code=403: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Request has expired</Message><X-Amz-Expires>600</X-Amz-Expires><Expires>2024-03-05T08:13:00Z</Expires><ServerTime>2024-03-05T16:03:02Z</ServerTime><RequestId>1Z3AYSR4JHAZ8DHR</RequestId><HostId>DWubBCiRJXWVyZNlKw0IgyaTXInwQjtHLT/qhqpXu96VlcF6d8fVcUl5jxEuQkveS8pyJNuvrXqmwZfpEGexcg==</HostId></Error>
instrument.js:191 sync-local->remote unknown: #error {:message Promise error, :data {:error :promise-error}, :cause #object[Error Error: Error invoking remote method 'main': Error: Can not upload temp file, code=403: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Request has expired</Message><X-Amz-Expires>600</X-Amz-Expires><Expires>2024-03-05T08:13:00Z</Expires><ServerTime>2024-03-05T16:03:02Z</ServerTime><RequestId>1Z3AYSR4JHAZ8DHR</RequestId><HostId>DWubBCiRJXWVyZNlKw0IgyaTXInwQjtHLT/qhqpXu96VlcF6d8fVcUl5jxEuQkveS8pyJNuvrXqmwZfpEGexcg==</HostId></Error>]}
instrument.js:191 [SyncManager] idle -> idle :reason nil :local-txid 252 :args nil :now 2024-03-05T08:03:01.558Z
instrument.js:191 TypeError: Cannot read properties of undefined (reading 'data')
    at sentry.cljs:45:40
    at sentry.cljs:45:26
    at request.js:1:1
    at Array.<anonymous> (promisebuffer.js:47:14)
    at promisebuffer.js:33:17
    at Array.forEach (<anonymous>)
    at y._executeHandlers (promisebuffer.js:93:9)
    at promisebuffer.js:52:16
    at y (promisebuffer.js:35:20)
    at y.then (promisebuffer.js:40:20)
    at A._processEvent (baseclient.js:1:63)
    at A._captureEvent (baseclient.js:461:62)
    at baseclient.js:137:26
    at Array.<anonymous> (promisebuffer.js:47:14)
    at promisebuffer.js:33:17
    at Array.forEach (<anonymous>)
    at y._executeHandlers (promisebuffer.js:93:9)
    at promisebuffer.js:52:16
    at y (promisebuffer.js:35:20)
    at y.then (promisebuffer.js:40:20)
    at A.captureException (baseclient.js:132:16)
    at g._invokeClient (hub.js:467:78)
    at g.captureException (hub.js:204:29)
    at w (index.js:169:1)
    at Q.captureException (index.js:1:63)
    at events.cljs:456:1
    at $APP.$JSCompiler_prototypeAlias$$.$cljs$core$IFn$_invoke$arity$1$ (core.cljs:11345:18)
    at $switch__29379__auto__$jscomp$187$$ (events.cljs:973:5)
    at Function.$frontend$handler$events$run_BANG__$_state_machine__29380__auto____1$$ [as $cljs$core$IFn$_invoke$arity$1$] (events.cljs:973:5)
    at $cljs$core$async$impl$ioc_helpers$run_state_machine_wrapped$$ (ioc_helpers.cljs:46:3)
    at ioc_helpers.cljs:53:3
    at channels.cljs:65:72
    at $cljs$core$async$impl$dispatch$process_messages$$ (dispatch.cljs:29:23)
    at $channel$jscomp$1$$.port1.onmessage (nexttick.js:190:5)

Are you willing to submit a PR? If you know how to fix the bug.

andelf commented 8 months ago

Could you check the recent nightly release? I've added a simple fix. https://github.com/logseq/logseq/releases/tag/nightly

stucash commented 8 months ago

@andelf Thanks for the swift response; I tried the alpha+nightly 0.10.7 AppImage; unforunately it didn't fix the issue on my end.

I reproduced it with exactly the same steps detailed earlier, but with the nightly version.

Looks like this time a different error manifested; the S3 token is expired.

Here's the new error log from dev console:

[01b1f34f-8489-4276-9d78-95b5af6a68db] setting sync age-encryption passphrase...
instrument.js:191 [SyncManager] need-password -> idle :reason nil :local-txid 252 :args nil :now 2024-03-06T14:20:38.530Z
instrument.js:191 [SyncManager] idle -> remote->local-full-sync :reason nil :local-txid 252 :args nil :now 2024-03-06T14:20:38.531Z
instrument.js:191 :frontend.fs.sync/diff-file-metadata-sets-elapsed-time 7.399999998509884 ms
instrument.js:191 [full-sync(remote->local)] 1 files need to sync
instrument.js:191 update-local-files 01b1f34f-8489-4276-9d78-95b5af6a68db /home/demo/Live-usb-storage/PKM/Logseq (pages/读书___笔记___马拉松训练指南.md)
instrument.js:191 [SyncManager] remote->local-full-sync -> idle :reason nil :local-txid 252 :args nil :now 2024-03-06T14:20:39.568Z
instrument.js:191 [SyncManager] idle -> local->remote-full-sync :reason nil :local-txid 252 :args nil :now 2024-03-06T14:20:39.569Z
instrument.js:191 [full-sync(local->remote)] 26 files need to sync and 0 local files need to delete
instrument.js:191 :sync-local->remote change [pages/读书___疑问___马拉松训练指南.md assets/strength_stage_three_beginner.jpeg assets/拉伸动作2.jpeg assets/训练量表格3.jpeg assets/training_cycle_two.jpeg assets/拉伸动作1.jpeg assets/training_cycle_three.jpeg assets/strength_stage_three_advanced.jpeg assets/拉伸动作5.jpeg assets/strength_stage_one_advanced.jpeg journals/2024_03_04.md assets/训练量表格2.jpeg assets/单边动作1.jpeg assets/strength_stage_two_2.jpeg assets/training_cycle_one.jpeg assets/strength_stage_two_1.jpeg assets/训练量表格1.jpeg assets/training_cycle_four.jpeg assets/strength_stage_two_intensity.jpeg assets/Electric-Blues-DanielsTables-3-01-02.xlsm]
instrument.js:191 Parsing start:  pages/读书___笔记___马拉松训练指南.md
instrument.js:191 Parsing finished:  pages/读书___笔记___马拉松训练指南.md
instrument.js:191 rsapi error: Error: Error invoking remote method 'main': Error: ExpiredToken: s3 token expired
instrument.js:191 sync-local->remote unknown: #error {:message Promise error, :data {:error :promise-error}, :cause #object[Error Error: Error invoking remote method 'main': Error: ExpiredToken: s3 token expired]}
instrument.js:191 [SyncManager] local->remote-full-sync -> idle :reason nil :local-txid 252 :args nil :now 2024-03-06T14:20:42.218Z
instrument.js:191 TypeError: Cannot read properties of undefined (reading 'data')
    at sentry.cljs:45:40
    at sentry.cljs:45:26
    at request.js:1:1
    at Array.<anonymous> (promisebuffer.js:47:14)
    at promisebuffer.js:33:17
    at Array.forEach (<anonymous>)
    at y._executeHandlers (promisebuffer.js:93:9)
    at promisebuffer.js:52:16
    at new y (promisebuffer.js:35:20)
    at y.then (promisebuffer.js:40:20)
    at A._processEvent (baseclient.js:1:63)
    at A._captureEvent (baseclient.js:461:62)
    at baseclient.js:137:26
    at Array.<anonymous> (promisebuffer.js:47:14)
    at promisebuffer.js:33:17
    at Array.forEach (<anonymous>)
    at y._executeHandlers (promisebuffer.js:93:9)
    at promisebuffer.js:52:16
    at new y (promisebuffer.js:35:20)
    at y.then (promisebuffer.js:40:20)
    at A.captureException (baseclient.js:132:16)
    at g._invokeClient (hub.js:467:78)
    at g.captureException (hub.js:204:29)
    at w (index.js:169:1)
    at Q.captureException (index.js:1:63)
    at events.cljs:456:1
    at $APP.$JSCompiler_prototypeAlias$$.$cljs$core$IFn$_invoke$arity$1$ (core.cljs:11345:18)
    at $switch__29499__auto__$jscomp$187$$ (events.cljs:973:5)
    at Function.$frontend$handler$events$run_BANG__$_state_machine__29500__auto____1$$ [as $cljs$core$IFn$_invoke$arity$1$] (events.cljs:973:5)
    at $cljs$core$async$impl$ioc_helpers$run_state_machine_wrapped$$ (ioc_helpers.cljs:46:3)
    at ioc_helpers.cljs:53:3
    at channels.cljs:65:72
    at $cljs$core$async$impl$dispatch$process_messages$$ (dispatch.cljs:29:23)
    at $channel$jscomp$1$$.port1.onmessage (nexttick.js:190:5)
atYuguo commented 7 months ago

I have the same problem, looking forward to a solution, thanks.

github-actions[bot] commented 1 month ago

Hi There! 👋

We haven't seen any activity on this issue in a while :sleeping:, and we just wanted to make sure that it's still relevant. If you're still experiencing this issue, you might find it helpful to update to the latest version of Logseq. The latest version includes bug fixes and new features that may help to resolve this issue, and you can download it from our website. If updating to the latest version doesn't help, please let us know by adding a comment 💬. We're here to help!

If the issue has been resolved or is no longer relevant, that's great news! 🎉 We'll go ahead and close this issue to keep our backlog organized. Please note that this issue will be closed automatically in 20 days if there is no further activity. If you need more time to resolve the issue or provide more information, please just let us know by adding a comment.

Access additional Logseq 🚀 resources:

Thanks for your contributions to Logseq! If you have any other issues or feature requests, please don't hesitate to let us know. We always welcome pull requests too!