[0.23.17] In latest version, any note update will trigger resynchronisation of all attachments in all devices

flyingnobita commented 1 month ago

Hi! Thanks for your great work! Just noticed something strange in the latest version that I like to report and wondering if you can help me with. Thanks in advance!

Abstract

I just updated to the newest version of 0.23.17. Syntonisation was working fine in previous version.

Whenever a change to a note is made, all attachments (mostly pictures and pdfs) are being created as "new note", re-synced, marked as conflict, and finally merged automatically as no change was detected.

Expected behaviour

Synchronisation of changed files only (as it was in the previously version)

Actually happened

make any change in a note in one of my device

For only attachments (where they are all stored in /zz_attachments/), the following message would appear for (seemingly) all files in /zz_atachments/:

7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: (new note)...
7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: Conflicted revision has been deleted, but there were more conflicts...
7/15/2024, 11:32:18 PM->Processing: zz_attachments/05f18db3dcfc4352f02f50651aee725f.gif: Done

This process takes ~5-10 mins where new lines are created in the logs non-stop (a sign to me that suggests the same thing was probably happening to all attachment files)
It occurs in all my other devices (~5) even though no changes were made there (so just syncing changes from my first device)
At the end, all files synchronised and no files are incorrectly deleted (fortunately!)
When this happens on my iPhone, sometimes Obsidian would restart during the above and continue after restart finishes
If I make a change in another device, the same strange behaviour above would happen to the rest of the devices, suggesting that it is not a platform or device issue

Reproducing procedure

Update to version 0.23.17
make change in device 1
Start synchronization on other devices

Report from the LiveSync

If the above isn't sufficient, I can follow up with more information, just let me know.

ZHmao commented 1 month ago

+1, I have the same issue.

vrtmrz commented 4 weeks ago

Thank you for your detailed information! However, I regret to say that I still cannot reproduce it.

Would you share with me a list of your other plug-ins (This can be seen by Show debug info from the command palette), And a configuration report of Self-hosted LiveSync (This can be dumped from the Make report button on the Hatch pane on the setting dialogue), please?

And, if nothing has changed, it should not be saved. Possibly it looks like the Conflict has been successfully resolved, but in fact, it has not. Does Pick a file to resolve conflict show the files? If so, rebuilding the database may solve the problem for now.

flyingnobita commented 4 weeks ago

Thank you for your reply.

I just tried rebuilding the database and then fetching it from other devices, but it didn't help and the same behaviour I reported occurred unfortunately.

Does Pick a file to resolve conflict show the files?

No files are shown.

Other Plugins

``` Plugins installed: 59 Plugins enabled: 57 1: Git v2.25.0 2: Self-hosted LiveSync v0.23.20 3: Auto Link Title v1.5.4 4: Better Word Count v0.10.1 5: Link Favicons v1.8.4 6: Plugin Update Tracker v1.5.2 7: Recent Files v1.4.1 8: Vault Statistics v0.1.3 9: Style Settings v1.0.8 10: Settings Search v1.3.10 11: Find orphaned files and broken links v1.10.0 12: Trash Explorer v1.2.2 13: Dataview v0.5.67 14: Natural Language Dates v0.6.2 15: Advanced Tables v0.21.0 16: Optimize Canvas Connections v1.0.0 17: Supercharged Links v0.12.1 18: Tag Wrangler v0.6.1 19: TagFolder v0.18.7 20: Safe Filename Linter v1.1.4 21: Templater v2.4.1 22: Note Refactor v1.8.2 23: Tab Switcher v1.4.0 24: Emoji Toolbar v0.4.1 25: Footnote Shortcut v0.1.3 26: Hotkeys for specific files v1.4.1 27: Smarter Markdown Hotkeys v1.16.4 28: Todoist Sync v1.13.0 29: Tasks v7.6.1 30: Calendar v1.5.10 31: Periodic Notes v0.0.17 32: QuickAdd v1.11.0 33: Hider v1.4.0 34: Keyboard Analyzer v0.1.3 35: Task Marker v0.5.1 36: Workspaces Plus v0.3.3 37: Excalidraw v2.2.13 38: Image Converter v1.1.7 39: File Diff v1.1.1 40: Quiet Outline v0.3.36 41: Heading Shifter v1.5.1 42: Editor Syntax Highlight v0.1.3 43: Commander v0.5.1 44: Number Headings v1.16.0 45: Linter v1.25.0 46: Projects v1.17.4 47: ProgressBar v0.6.0 48: Homepage v4.0.3 49: Outliner v4.8.1 50: Home tab v1.2.1 51: Image Toolkit v1.4.1 52: Minimal Theme Settings v8.0.2 53: Zotero Integration v3.1.8 54: Mononote v1.2.2 55: Lapel v0.1.5 56: Omnisearch v1.24.0 57: Readwise Official v2.0.1 ```

LiveSync Report

``` ---- Obsidian info ---- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.4.13 Chrome/114.0.5735.289 Electron/25.8.1 Safari/537.36 ---- remote config ---- cluster: n: "1" cors: credentials: "true" headers: accept, authorization, content-type, origin, referer max_age: "3600" methods: GET, PUT, POST, HEAD, DELETE origins: app://obsidian.md,capacitor://localhost,http://localhost chttpd: bind_address: 0.0.0.0 enable_cors: "true" max_http_request_size: "4294967296" port: "5984" require_valid_user: "true" admins: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 vendor: name: The Apache Software Foundation feature_flags: partitioned||*: "true" chttpd_auth: authentication_redirect: /_utils/session.html hash_algorithms: sha256, sha require_valid_user: "true" secret: b3e793bd47164192d8c353ecaeb12e5d indexers: couch_mrview: "true" prometheus: additional_port: "false" bind_address: 127.0.0.1 port: "17986" httpd: WWW-Authenticate: Basic realm="couchdb" bind_address: 127.0.0.1 enable_cors: "true" port: "5986" smoosh: state_dir: ./data couch_httpd_auth: authentication_db: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 secret: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 authentication_redirect: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchdb_engines: couch: couch_bt_engine couchdb: database_dir: ./data max_document_size: "50000000" single_node: "true" uuid: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 view_index_dir: ./data ---- Plug-in config --- version:0.23.20 remoteType: "" useCustomRequestHandler: false couchDB_URI: self-hosted(HTTP) couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 liveSync: true syncOnSave: false syncOnStart: false savingDelay: 200 lessInformationInLog: true gcDelay: 0 versionUpFlash: "" minimumChunkSize: 20 longLineThreshold: 250 showVerboseLog: false suspendFileWatching: false trashInsteadDelete: true periodicReplication: false periodicReplicationInterval: 60 syncOnFileOpen: false encrypt: true passphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 usePathObfuscation: false doNotDeleteFolder: false resolveConflictsByNewerFile: false batchSave: false batchSaveMinimumDelay: 5 batchSaveMaximumDelay: 60 deviceAndVaultName: MBP usePluginSettings: false showOwnPlugins: false showStatusOnEditor: false showStatusOnStatusbar: true showOnlyIconsOnEditor: false usePluginSync: false autoSweepPlugins: false autoSweepPluginsPeriodic: false notifyPluginOrSettingUpdated: false checkIntegrityOnSave: false batch_size: 50 batches_limit: 40 useHistory: true disableRequestURI: true skipOlderFilesOnSync: true checkConflictOnlyOnOpen: false showMergeDialogOnlyOnActive: false syncInternalFiles: false syncInternalFilesBeforeReplication: false syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/ syncInternalFilesInterval: 60 additionalSuffixOfDatabaseName: "" ignoreVersionCheck: false lastReadUpdates: 23 deleteMetadataOfDeletedFiles: false syncIgnoreRegEx: "" syncOnlyRegEx: "" customChunkSize: 0 readChunksOnline: true watchInternalFileChanges: true automaticallyDeleteMetadataOfDeletedFiles: 0 disableMarkdownAutoMerge: false writeDocumentsIfConflicted: false useDynamicIterationCount: false syncAfterMerge: false configPassphraseStore: "" encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 permitEmptyPassphrase: false useIndexedDBAdapter: true useTimeouts: false writeLogToTheFile: false doNotPaceReplication: false hashCacheMaxCount: 300 hashCacheMaxAmount: 50 concurrencyOfReadChunksOnline: 100 minimumIntervalOfReadChunksOnline: 333 hashAlg: xxhash64 suspendParseReplicationResult: false doNotSuspendOnFetching: false useIgnoreFiles: false ignoreFiles: .gitignore syncOnEditorSave: false pluginSyncExtendedSetting: {} syncMaxSizeInMB: 50 settingSyncFile: "" writeCredentialsForSettingSync: false notifyAllSettingSyncFile: false isConfigured: true settingVersion: 0 enableCompression: false accessKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 bucket: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(0 letters) endpoint: Not configured or AWS region: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(4 letters) secretKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 useEden: false maxChunksInEden: 10 maxTotalLengthInEden: 1024 maxAgeInEden: 10 disableCheckingConfigMismatch: false displayLanguage: "" enableChunkSplitterV2: false disableWorkerForGeneratingChunks: false processSmallFilesInUIThread: false notifyThresholdOfRemoteStorageSize: 0 usePluginSyncV2: false usePluginEtc: false useV1: true configPassphrase: "" preset: "" syncMode: LIVESYNC dummy: 0 ```

I hope this helps, thank you!

Laharah commented 2 weeks ago

+1 Here too.

I've rebuilt my remote and all my local databases and the issue persists. I've also removed any orphan attachments just in case. I haven't had a chance to dig into things and see what's going on yet, but I can confirm the issue. I'll see if I can produce a Minimal Reproducible Example once I've got some free time.

vrtmrz / obsidian-livesync