vrtmrz / obsidian-livesync

MIT License
5.14k stars 167 forks source link

Synchronisation never finishes when using a proxy #466

Closed Estyms closed 4 months ago

Estyms commented 4 months ago

Abstract

When using a proxy to sync, it keeps sending request and never finishes the synchronization.

Expected behaviour

Actually happened

Reproducing procedure

  1. Configure LiveSync as in the attached material.
  2. Click the replication button on the ribbon.
  3. Synchronising has begun.
  4. It never finishes

Report materials

Report from the LiveSync

Report from hatch ``` ---- Obsidian info ---- Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.5.12 Chrome/120.0.6099.283 Electron/28.2.3 Safari/537.36 ---- remote config ---- cors: credentials: "true" origins: app://obsidian.md,capacitor://localhost,http://localhost chttpd: authentication_handlers: "{chttpd_auth, proxy_authentication_handler}, {chttpd_auth, default_authentication_handler}" bind_address: any max_http_request_size: "4294967296" port: "5984" require_valid_user: "true" admins: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 vendor: name: The Apache Software Foundation feature_flags: partitioned||*: "true" chttpd_auth: hash_algorithms: sha256, sha proxy_use_secret: "true" require_valid_user: "true" secret: 2cffa5ece421c9d2719e5d798bd356dc52d1d72e indexers: couch_mrview: "true" prometheus: additional_port: "false" bind_address: 127.0.0.1 port: "17986" httpd: WWW-Authenticate: Basic realm="couchdb" bind_address: 127.0.0.1 enable_cors: "true" port: "5986" smoosh: state_dir: ./data couch_httpd_auth: authentication_db: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 secret: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 authentication_redirect: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 couchdb_engines: couch: couch_bt_engine couchdb: database_dir: ./data max_document_size: "50000000" uuid: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 view_index_dir: ./data ---- Plug-in config --- version:0.23.17 remoteType: "" useCustomRequestHandler: false couchDB_URI: self-hosted(HTTP) couchDB_USER: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 liveSync: false syncOnSave: false syncOnStart: false savingDelay: 200 lessInformationInLog: false gcDelay: 0 versionUpFlash: "" minimumChunkSize: 20 longLineThreshold: 250 showVerboseLog: false suspendFileWatching: false trashInsteadDelete: true periodicReplication: false periodicReplicationInterval: 10 syncOnFileOpen: false encrypt: false passphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 usePathObfuscation: false doNotDeleteFolder: false resolveConflictsByNewerFile: true batchSave: false batchSaveMinimumDelay: 5 batchSaveMaximumDelay: 60 deviceAndVaultName: minastirith usePluginSettings: false showOwnPlugins: false showStatusOnEditor: true showStatusOnStatusbar: true showOnlyIconsOnEditor: false usePluginSync: false autoSweepPlugins: false autoSweepPluginsPeriodic: false notifyPluginOrSettingUpdated: false checkIntegrityOnSave: false batch_size: 25 batches_limit: 25 useHistory: true disableRequestURI: true skipOlderFilesOnSync: true checkConflictOnlyOnOpen: false showMergeDialogOnlyOnActive: false syncInternalFiles: false syncInternalFilesBeforeReplication: false syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/ syncInternalFilesInterval: 60 additionalSuffixOfDatabaseName: 7cc60edc2e230f07 ignoreVersionCheck: false lastReadUpdates: 23 deleteMetadataOfDeletedFiles: false syncIgnoreRegEx: "" syncOnlyRegEx: "" customChunkSize: 50 readChunksOnline: false watchInternalFileChanges: true automaticallyDeleteMetadataOfDeletedFiles: 0 disableMarkdownAutoMerge: false writeDocumentsIfConflicted: false useDynamicIterationCount: false syncAfterMerge: false configPassphraseStore: "" encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 permitEmptyPassphrase: false useIndexedDBAdapter: true useTimeouts: true writeLogToTheFile: false doNotPaceReplication: false hashCacheMaxCount: 300 hashCacheMaxAmount: 50 concurrencyOfReadChunksOnline: 30 minimumIntervalOfReadChunksOnline: 25 hashAlg: xxhash64 suspendParseReplicationResult: false doNotSuspendOnFetching: false useIgnoreFiles: false ignoreFiles: .gitignore syncOnEditorSave: false pluginSyncExtendedSetting: {} syncMaxSizeInMB: 50 settingSyncFile: "" writeCredentialsForSettingSync: false notifyAllSettingSyncFile: false isConfigured: true settingVersion: 0 enableCompression: false accessKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 bucket: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(0 letters) endpoint: Not configured or AWS region: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷(4 letters) secretKey: 𝑅𝐸𝐷𝐴𝐢𝑇𝐸𝐷 useEden: false maxChunksInEden: 10 maxTotalLengthInEden: 1024 maxAgeInEden: 10 disableCheckingConfigMismatch: false displayLanguage: "" enableChunkSplitterV2: false disableWorkerForGeneratingChunks: false processSmallFilesInUIThread: false notifyThresholdOfRemoteStorageSize: 0 configPassphrase: "" preset: "" syncMode: ONEVENTS dummy: 0 ```

Plug-in log

Plug-in log ``` CouchDB-7cc60edc2e230f07:14/07/2024 18:50:37->OneShot Sync begin... (sync) CouchDB-7cc60edc2e230f07:14/07/2024 18:50:37->Looking for the point last synchronized point. CouchDB-7cc60edc2e230f07:14/07/2024 18:50:37->Replication activated ```

Network log

In the following archive you have the proxyfied as well as the direct access to the couchdb instance har-files.zip

Screenshots

image

Proxy used

If it can help, the proxy used is the following one : keycloak-couchdb-proxy The implementation seems right as, as you can see in the network logs, the responses are the same.

Estyms commented 4 months ago

After tinkering a bit, I noticed that my proxy wasn't forwarding the query parameters, adding them back in the logic fixed the issue