Live sync not working, has to manually call "Replicate now" to sync

TheNetAdmin commented 1 year ago

Thank you for taking the time to report this issue! To improve the process, I would like to ask you to let me know the information in advance.

All instructions and examples, and empty entries can be deleted. Just for your information, a filled example is also written.

Abstract

The livesync does not work on a fresh environment (new obsidian, new self-hosted CouchDB). I have to manually execute the command palette Self-hosted LiveSync: Replicate now to sync documents.

Here's an example

Set up CouchDB using Docker
Set up Obsidian livesync plug in
Create a file in Obsidian, called 2nd new file
The Obsidian client-side log says 9/19/2023, 1:40:49 PM->DB <- STORAGE (plain) 2nd new file.md (I did not manually execute Self-hosted LiveSync: Replicate now, this log line just automatically showed up)

Expected behaviour

The 2nd new file.md should be updated into couchdb

Actually happened

CouchDB server side (shown in the below screenshot) shows no such 2nd new file.md file, even though the Obsidian client-side log says the upload is finished.
Note the CouchDB has a New file from lab pc.md file, which is manually synced by executing Obsidian command palette Self-hosted LiveSync: Replicate now, and it happens before the creation of 2nd new file.md

Screenshot_2023-09-19_13-50-02

Reproducing procedure

Shown in the abstract

Report materials

If the information is not available, do not hesitate to report it as it is. You can also of course omit it if you think this is indeed unnecessary. If it is necessary, I will ask you.

Report from the LiveSync

For more information, please refer to Making the report.

Report from hatch

``` ----remote config---- cors: credentials: "true" headers: accept, authorization, content-type, origin, referer max_age: "3600" methods: GET,PUT,POST,HEAD,DELETE origins: app://obsidian.md,capacitor://localhost,http://localhost chttpd: bind_address: any enable_cors: "true" max_http_request_size: "4294967296" port: "5984" require_valid_user: "true" admins: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 vendor: name: The Apache Software Foundation feature_flags: partitioned||*: "true" chttpd_auth: authentication_redirect: /_utils/session.html hash_algorithms: sha256, sha require_valid_user: "true" indexers: couch_mrview: "true" prometheus: additional_port: "false" bind_address: 127.0.0.1 port: "17986" httpd: WWW-Authenticate: Basic realm="couchdb" bind_address: 0.0.0.0 enable_cors: "true" port: "5986" smoosh: state_dir: ./data couch_httpd_auth: authentication_db: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 secret: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 authentication_redirect: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchdb_engines: couch: couch_bt_engine couchdb: database_dir: ./data max_document_size: "50000000" uuid: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 view_index_dir: ./data ---- Plug-in config --- couchDB_URI: self-hosted couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 liveSync: true syncOnSave: false syncOnStart: false savingDelay: 200 lessInformationInLog: false gcDelay: 300 versionUpFlash: "" minimumChunkSize: 20 longLineThreshold: 250 showVerboseLog: false suspendFileWatching: false trashInsteadDelete: true periodicReplication: false periodicReplicationInterval: 60 syncOnFileOpen: false encrypt: false passphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 usePathObfuscation: false doNotDeleteFolder: false resolveConflictsByNewerFile: false batchSave: false deviceAndVaultName: "" usePluginSettings: false showOwnPlugins: false showStatusOnEditor: true usePluginSync: false autoSweepPlugins: false autoSweepPluginsPeriodic: false notifyPluginOrSettingUpdated: false checkIntegrityOnSave: false batch_size: 50 batches_limit: 40 useHistory: false disableRequestURI: false skipOlderFilesOnSync: true checkConflictOnlyOnOpen: false syncInternalFiles: false syncInternalFilesBeforeReplication: false syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/ syncInternalFilesInterval: 60 additionalSuffixOfDatabaseName: "" ignoreVersionCheck: false lastReadUpdates: 19 deleteMetadataOfDeletedFiles: false syncIgnoreRegEx: "" syncOnlyRegEx: "" customChunkSize: 100 readChunksOnline: true watchInternalFileChanges: true automaticallyDeleteMetadataOfDeletedFiles: 0 disableMarkdownAutoMerge: false writeDocumentsIfConflicted: false useDynamicIterationCount: false syncAfterMerge: false configPassphraseStore: "" encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 permitEmptyPassphrase: false useIndexedDBAdapter: true useTimeouts: false writeLogToTheFile: false doNotPaceReplication: false hashCacheMaxCount: 300 hashCacheMaxAmount: 50 concurrencyOfReadChunksOnline: 100 minimumIntervalOfReadChunksOnline: 333 hashAlg: xxhash64 suspendParseReplicationResult: false doNotSuspendOnFetching: false useIgnoreFiles: false ignoreFiles: .gitignore syncOnEditorSave: false pluginSyncExtendedSetting: {} ```

Obsidian debug info

Debug info

``` ```

Plug-in log

We can see the log by tapping the Document box icon. If you noticed something suspicious, please let me know. Note: Please enable Verbose Log. For detail, refer to Logging, please.

Plug-in log

``` 9/19/2023, 1:24:41 PM->Content saved:Untitled.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:24:41 PM->DB <- STORAGE (plain) Untitled.md 9/19/2023, 1:24:46 PM->Entry removed:Untitled.md (Untitled.md-2-345bac4c16cd42b097c0fea3a0b4b130) 9/19/2023, 1:24:46 PM->Content saved:New file from lab pc.md ,chunks: 1 (new:0, skip:2, cache:1) 9/19/2023, 1:24:46 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:25:05 PM->OneShot Sync begin... (sync) 9/19/2023, 1:25:05 PM->Replication activated 9/19/2023, 1:25:05 PM->Replication completed 9/19/2023, 1:26:36 PM->Content saved:New file from lab pc.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:26:36 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:31:45 PM->Content saved:New file from lab pc.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:31:45 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:33:33 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:2, skip:0, cache:0) 9/19/2023, 1:33:33 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:33:35 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:1, skip:2, cache:1) 9/19/2023, 1:33:36 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:33:38 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:1, skip:2, cache:1) 9/19/2023, 1:33:38 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:35:10 PM->Content saved:New file from lab pc.md ,chunks: 12 (new:9, skip:2, cache:1) 9/19/2023, 1:35:10 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:35:13 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:0, skip:4, cache:2) 9/19/2023, 1:35:13 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:38:56 PM->Database closed for reset Database. 9/19/2023, 1:38:56 PM->Opening Database... 9/19/2023, 1:38:56 PM->Database is now ready. 9/19/2023, 1:38:56 PM->Local Database Reset 9/19/2023, 1:38:56 PM->Database closed (by close) 9/19/2023, 1:38:56 PM->Waiting for ready... 9/19/2023, 1:38:56 PM->Error:database is closed 9/19/2023, 1:38:56 PM->Opening Database... 9/19/2023, 1:38:57 PM->Database is now ready. 9/19/2023, 1:38:57 PM->Initialize and checking database files 9/19/2023, 1:38:57 PM->Checking deleted files 9/19/2023, 1:38:57 PM->Database looks empty, save files as initial sync data 9/19/2023, 1:38:57 PM->Updating database by new files 9/19/2023, 1:38:57 PM->UPDATE DATABASE 9/19/2023, 1:38:57 PM->UPDATE DATABASE New file from lab pc.md 9/19/2023, 1:38:57 PM->Content saved:New file from lab pc.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:38:57 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:38:57 PM->UPDATE DATABASE: PASS:1, FAILED:0 9/19/2023, 1:38:57 PM->Initialized, NOW TRACKING! 9/19/2023, 1:39:04 PM->Configuration has been flushed, please open it again 9/19/2023, 1:39:32 PM->Error:Request failed, status 404 9/19/2023, 1:39:32 PM->Connected to obsidian : Snipped 9/19/2023, 1:39:48 PM->Synchronization setting configured as LiveSync. 9/19/2023, 1:39:48 PM->Database closed for reset Database. 9/19/2023, 1:39:48 PM->Opening Database... 9/19/2023, 1:39:48 PM->Database is now ready. 9/19/2023, 1:39:48 PM->Local Database Reset 9/19/2023, 1:39:48 PM->Database closed (by close) 9/19/2023, 1:39:48 PM->Waiting for ready... 9/19/2023, 1:39:48 PM->Error:database is closed 9/19/2023, 1:39:48 PM->Opening Database... 9/19/2023, 1:39:48 PM->Database is now ready. 9/19/2023, 1:39:48 PM->Initializing 9/19/2023, 1:39:48 PM->Initialize and checking database files 9/19/2023, 1:39:48 PM->Checking deleted files 9/19/2023, 1:39:48 PM->Database looks empty, save files as initial sync data 9/19/2023, 1:39:48 PM->Updating database by new files 9/19/2023, 1:39:48 PM->UPDATE DATABASE 9/19/2023, 1:39:48 PM->UPDATE DATABASE New file from lab pc.md 9/19/2023, 1:39:48 PM->Content saved:New file from lab pc.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:39:48 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:39:48 PM->UPDATE DATABASE: PASS:1, FAILED:0 9/19/2023, 1:39:48 PM->Initialized, NOW TRACKING! 9/19/2023, 1:39:48 PM->Initialize done! 9/19/2023, 1:39:48 PM->OneShot Sync begin... (sync) 9/19/2023, 1:39:48 PM->Error:Request failed, status 404 9/19/2023, 1:39:48 PM->Error:Request failed, status 404 9/19/2023, 1:39:48 PM->Error:Request failed, status 404 9/19/2023, 1:39:49 PM->Error:Request failed, status 404 9/19/2023, 1:39:49 PM->Looking for the point last synchronized point. 9/19/2023, 1:39:49 PM->Error:Request failed, status 404 9/19/2023, 1:39:49 PM->Replication activated 9/19/2023, 1:39:49 PM->↑3 (LIVE) ↓0 9/19/2023, 1:39:49 PM->Error:Request failed, status 404 9/19/2023, 1:39:49 PM->Replication activated 9/19/2023, 1:39:49 PM->↑3 (LIVE) ↓2 (LIVE) 9/19/2023, 1:39:49 PM->Replication completed 9/19/2023, 1:39:49 PM->All done! Please set up subsequent devices with 'Copy setup URI' and 'Open setup URI'. 9/19/2023, 1:39:54 PM->Setup URI copied to clipboard 9/19/2023, 1:40:12 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:2, skip:0, cache:0) 9/19/2023, 1:40:12 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:40:14 PM->Content saved:New file from lab pc.md ,chunks: 2 (new:1, skip:2, cache:1) 9/19/2023, 1:40:14 PM->DB <- STORAGE (plain) New file from lab pc.md 9/19/2023, 1:40:45 PM->Content saved:Untitled.md ,chunks: 1 (new:1, skip:0, cache:0) 9/19/2023, 1:40:45 PM->DB <- STORAGE (plain) Untitled.md 9/19/2023, 1:40:49 PM->Entry removed:Untitled.md (Untitled.md-2-2ae9ce8650ba4d9fb400cda3ab0ea421) 9/19/2023, 1:40:49 PM->Content saved:2nd new file.md ,chunks: 1 (new:0, skip:2, cache:1) 9/19/2023, 1:40:49 PM->DB <- STORAGE (plain) 2nd new file.md : Snipped 9/19/2023, 1:44:21 PM->Information has been copied to clipboard ```

Network log

Network logs displayed in DevTools will possibly help with connection-related issues. To capture that, please refer to DevTools.

Screenshots

If applicable, please add screenshots to help explain your problem.

Other information, insights and intuition.

Please provide any additional context or information about the problem.

StrangePeanut commented 1 year ago

Experiencing the same here. The issue seems to stem from the latest release, 0.19.20. I'm not sure if it's safe to downgrade.

vrtmrz commented 1 year ago

I will address it later! but, please change the password of your CouchDB. (Really sorry for writing that into the log, even if it has been hashed).

TheNetAdmin commented 1 year ago

Ah I will... didn't see that but it's a local instance never exposed to the public so it's gonna be fine, thanks!

TheFiZi commented 1 year ago

0.19.22 solved this for me

vrtmrz commented 1 year ago

Sorry for being late for making notice about this! @TheNetAdmin, I am very relieved to hear that! Since v0.19.22, our configuration of remote CouchDB will be dumped only on the button by the Hatch pane. And, as @TheFiZi already mentioned, this problem has also been fixed at v0.19.22. Thank you for testing it, If @TheNetAdmin also have no problems, I would be grateful if you could close it.

TheNetAdmin commented 1 year ago

Yeah I tried the new version and it seems fine to me, we can close the issue for now.

v3i1y commented 6 months ago

I am on 0.22.14 and still experiencing this. Have to manually click sync now for files to be uploaded and fetched, even then some files are not synced properly, have to edit the file to make them dirty to make them sync sometimes.

vrtmrz commented 5 months ago

@v3i1y Thank you for reporting this! Reproduced! It seems to have relapsed. Unfortunately, I have not noticed because it is not apparent when Hidden File or Customisation Sync has been enabled. I will address it later!

qlibp commented 4 months ago

@vrtmrz confirm that 0.23.6 version works as expected.

One more question, can we start obsidian at the back(I mean without focusing on it) and make livesync to sync editions made by other editors, like vim/vscode?

vrtmrz / obsidian-livesync