Replication errors between Mac / iPad / iPhone: TypeError, AbortError

kuyper commented 4 months ago

Abstract

Livesync throws TypeError and AbortError when trying to sync between Mac / iPad / iPhone.

Expected behaviour

Synchronisation ends with the message Replication completed
Everything synchronised

Actually happened

Initial synch of Mac with CouchDB works.

After adding synch to iPad, the following happens on the iPad:

HTTP:Post (xxx) to:/_bulk_docs -> failed
TypeError: Load failed

I also see

HTTP:GET to:/_changes?style=all_docs&feed=.... -> false
AbortError:Fetch is aborted

Reproducing procedure

Start with a fresh database in CouchDB
Setup a new vault on Mac
Enable community plugins and install livesync
Configure livesync to use the fresh database
Keep hidden file sync disabled
Setup new vault on iPad, install livesync
Configure livesync to use the database that has just been created
Keep hidden files disabled
Wait until first sync is done (which sometimes takes a while)
Enable customisation sync

--> Error start appearing in the logs

My feeling is that sync is generally unstable with this version. I'm using 0.21 with Mac / iPhone / iPad with three different vaults in a "production" setting and it works well. But with 0.22 I haven't been able to get it to run.

Report materials

If the information is not available, do not hesitate to report it as it is. You can also of course omit it if you think this is indeed unnecessary. If it is necessary, I will ask you.

Report from the LiveSync

For more information, please refer to Making the report.

Report from hatch

``` ---- Obsidian info ---- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) ---- remote config ---- cors: credentials: "true" headers: accept, authorization, content-type, origin, referer max_age: "3600" methods: GET,PUT,POST,HEAD,DELETE origins: app://obsidian.md,capacitor://localhost,http://localhost chttpd: bind_address: any enable_cors: "true" max_http_request_size: "4294967296" port: "5984" require_valid_user: "true" admins: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" vendor: name: The Apache Software Foundation feature_flags: partitioned||*: "true" chttpd_auth: authentication_redirect: /_utils/session.html hash_algorithms: sha256, sha require_valid_user: "true" indexers: couch_mrview: "true" prometheus: additional_port: "false" bind_address: 127.0.0.1 port: "17986" httpd: WWW-Authenticate: Basic realm="couchdb" bind_address: 0.0.0.0 enable_cors: "true" port: "5986" smoosh: state_dir: ./data couch_httpd_auth: authentication_db: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" secret: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" authentication_redirect: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" couchdb_engines: couch: couch_bt_engine couchdb: database_dir: ./data max_document_size: "50000000" single_node: "true" uuid: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" view_index_dir: ./data ---- Plug-in config --- version:0.22.7 couchDB_URI: self-hosted couchDB_USER: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" couchDB_PASSWORD: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" couchDB_DBNAME: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" liveSync: true syncOnSave: false syncOnStart: false savingDelay: 200 lessInformationInLog: false gcDelay: 0 versionUpFlash: "" minimumChunkSize: 20 longLineThreshold: 250 showVerboseLog: true suspendFileWatching: false trashInsteadDelete: true periodicReplication: false periodicReplicationInterval: 60 syncOnFileOpen: false encrypt: false passphrase: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" usePathObfuscation: false doNotDeleteFolder: false resolveConflictsByNewerFile: false batchSave: false deviceAndVaultName: "" usePluginSettings: false showOwnPlugins: false showStatusOnEditor: true showOnlyIconsOnEditor: false usePluginSync: true autoSweepPlugins: false autoSweepPluginsPeriodic: false notifyPluginOrSettingUpdated: false checkIntegrityOnSave: false batch_size: 50 batches_limit: 40 useHistory: true disableRequestURI: true skipOlderFilesOnSync: true checkConflictOnlyOnOpen: false showMergeDialogOnlyOnActive: false syncInternalFiles: false syncInternalFilesBeforeReplication: false syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/ syncInternalFilesInterval: 60 additionalSuffixOfDatabaseName: documents/test3 ignoreVersionCheck: false lastReadUpdates: 0 deleteMetadataOfDeletedFiles: false syncIgnoreRegEx: "" syncOnlyRegEx: "" customChunkSize: 50 readChunksOnline: true watchInternalFileChanges: true automaticallyDeleteMetadataOfDeletedFiles: 0 disableMarkdownAutoMerge: false writeDocumentsIfConflicted: false useDynamicIterationCount: false syncAfterMerge: false configPassphraseStore: "" encryptedPassphrase: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" encryptedCouchDBConnection: "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷" permitEmptyPassphrase: false useIndexedDBAdapter: true useTimeouts: false writeLogToTheFile: false doNotPaceReplication: false hashCacheMaxCount: 300 hashCacheMaxAmount: 50 concurrencyOfReadChunksOnline: 100 minimumIntervalOfReadChunksOnline: 333 hashAlg: xxhash64 suspendParseReplicationResult: false doNotSuspendOnFetching: false useIgnoreFiles: false ignoreFiles: .gitignore syncOnEditorSave: false pluginSyncExtendedSetting: {} syncMaxSizeInMB: 50 settingSyncFile: "" writeCredentialsForSettingSync: false notifyAllSettingSyncFile: false isConfigured: true settingVersion: 0 ```

vrtmrz commented 4 months ago

Thank you for reporting the issue! And sorry for being delayed!

I have created the same configuration environment. However, unfortunately, I could not reproduce it.

If your CouchDB is hosted on HTTP instead of HTTPS, it might be the same problem as #385. It will be happy to allow me to report the latest news on that thread. If that is hosted on HTTPS, please let me know.

kuyper commented 3 months ago

My couchDB is hosted behind a reverse proxy that offers https access, has an official DNS entry and also valid letsencrypt certificates. I use nginx as reverse proxy.

kuyper commented 3 months ago

I've started from scratch with the newest version, a new vault, new DB and I get this

24.3.2024, 13:42:32->Replication activated
24.3.2024, 13:42:37->STORAGE --> DB:ix:desktop/CONFIG/workspace.json.md: (config) Done
24.3.2024, 13:42:39->HTTP:POST (1174270) to:/_bulk_docs -> failed
24.3.2024, 13:42:39->TypeError:Failed to fetch
24.3.2024, 13:42:39->Replication paused

vrtmrz commented 2 months ago

Thank you for the additional information! Your configuration which we can check is all ok. There may be some trouble with some network configurations.

Would you mind if I ask you to check the DevTools log of your iOS devices via your Mac? You can connect and check it as this document

innerself commented 2 months ago

I have the same problem on Ubuntu. Tried to re-fetch, rebuild. Nothing helps. What log can I give? couch_db.log

M-Davies commented 2 months ago

I'm also hitting this issue for my Android phone (my config is in #408 ). My log is very similar:

21/04/2024, 23:01:59->OneShot Sync begin... (sync)
21/04/2024, 23:01:59->Looking for the point last synchronized point.
21/04/2024, 23:02:00->Replication activated
21/04/2024, 23:02:00->HTTP:POST (23920717) to:/_bulk_docs -> failed
21/04/2024, 23:02:00->TypeError:Failed to fetch
21/04/2024, 23:02:00->Replication error
21/04/2024, 23:02:00->Error:Failed to fetch
21/04/2024, 23:02:00->Replication stopped.
21/04/2024, 23:02:00->Retry with lower batch size:27/27
21/04/2024, 23:02:00->OneShot Sync begin... (sync)
21/04/2024, 23:02:00->Looking for the point last synchronized point.
21/04/2024, 23:02:00->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1631150) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:16/16
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1527781) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:10/10
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:01->HTTP:POST (1518925) to:/_bulk_docs -> failed
21/04/2024, 23:02:01->TypeError:Failed to fetch
21/04/2024, 23:02:01->Replication error
21/04/2024, 23:02:01->Error:Failed to fetch
21/04/2024, 23:02:01->Replication stopped.
21/04/2024, 23:02:01->Retry with lower batch size:7/7
21/04/2024, 23:02:01->OneShot Sync begin... (sync)
21/04/2024, 23:02:01->Looking for the point last synchronized point.
21/04/2024, 23:02:01->Replication activated
21/04/2024, 23:02:02->HTTP:POST (1514053) to:/_bulk_docs -> failed
21/04/2024, 23:02:02->TypeError:Failed to fetch
21/04/2024, 23:02:02->Replication error
21/04/2024, 23:02:02->Error:Failed to fetch
21/04/2024, 23:02:02->Replication stopped.
21/04/2024, 23:02:02->Retry with lower batch size:6/6
21/04/2024, 23:02:02->OneShot Sync begin... (sync)
21/04/2024, 23:02:02->Looking for the point last synchronized point.
21/04/2024, 23:02:02->Replication activated
21/04/2024, 23:02:02->HTTP:POST (1462233) to:/_bulk_docs -> failed
21/04/2024, 23:02:02->TypeError:Failed to fetch
21/04/2024, 23:02:02->HTTP:POST (36) to:/_changes?style=all_docs&filter=_selector&since=48-g1AAAACLeJzLYWBgYMpgTmHgzcvPy09JdcjLz8gvLskBCScyJNX___8_K4M5USwXKMCeZJ5iaZJojq4Yh_Y8FiDJ0ACk_kNNkQKbYpRmaZlmlIauJwsAY1Eq9A&limit=6 -> failed
21/04/2024, 23:02:02->AbortError:The user aborted a request.
21/04/2024, 23:02:02->Replication error
21/04/2024, 23:02:02->Error:Failed to fetch
21/04/2024, 23:02:02->Replication stopped.
21/04/2024, 23:02:02->We can't replicate more lower value.

M-Davies commented 2 months ago

Found the answer for my issue #344

vrtmrz commented 1 month ago

Sorry for the absence! I am relieved to hear that the problems have been resolved. (If I have misunderstood something, please feel free to reopen the issue).

vrtmrz / obsidian-livesync