Hidden file sync: hidden files can't get removed

[foegra]

Livesync version - 24rc6

When hidden files are modified on desktop - they do not get transfered to Andoid. Instead - old hidden files from Android directly overwrite changes made on desktop. Here is the log from Android device:

10/24/2024, 8:41:10 AM->Replication activated 10/24/2024, 8:41:10 AM->[HiddenFileSync]  START :Applying hidden 1 files change 10/24/2024, 8:41:11 AM->[HiddenFileSync]  Scanning hidden files. 10/24/2024, 8:41:11 AM->Replication paused 10/24/2024, 8:41:15 AM->[HiddenFileSync]  Hidden files scanned: 0 files had been modified 10/24/2024, 8:41:15 AM->[HiddenFileSync]  DONE :Applying hidden 1 files change 10/24/2024, 8:41:15 AM->[HiddenFileSync]  START :Applying hidden 1 files change 10/24/2024, 8:41:15 AM->[HiddenFileSync]  Scanning hidden files. 10/24/2024, 8:41:19 AM->[HiddenFileSync]  Hidden files scanned: 0 files had been modified 10/24/2024, 8:41:19 AM->[HiddenFileSync]  DONE :Applying hidden 1 files change 10/24/2024, 8:41:19 AM->[HiddenFileSync]  START :Applying hidden 1 files change 10/24/2024, 8:41:19 AM->[HiddenFileSync]  Scanning hidden files. 10/24/2024, 8:41:23 AM->[HiddenFileSync]  Hidden files scanned: 0 files had been modified 10/24/2024, 8:41:23 AM->[HiddenFileSync]  DONE :Applying hidden 1 files change

[vrtmrz]

Thank you for the testing!

However, I could not reproduce it yet. Would you mind sharing the report which can be dumped on the hatch pane of the setting dialogue?

And at rc7, checking files between the local database and storage is now also able to handle hidden files. This is also can be performed on the hatch pane. (Verify and repair all files). Would you mind if I ask you to try it once, please?

[foegra]

Thank You for the update. Tried with rc7 - still the same. If plugin is removed on one device - it's not getting removed on another. After restart - it reappears on the device, where it was initially removed.

Seems problem is only when i remove plugin. When plugin is installed on one device - it gets installed on the other one as well. If plugin settings get modified - changes get synced well as well.

[vrtmrz]

I appreciate your patience! Thanks to your cooperation and the detailed log you provided, we have successfully reproduced the issue.

As you mentioned, the deletion of the hidden file sync had not been reflected in the storage. Consequently, upon the next startup, those files were resurrected from the storage.

This behaviour is designed for safety, to prevent database corruption (the files in storage overwrite the database on the hidden file sync if the plug-in is in doubt about the decision). Thus, we should not have lost our chance; however, indeed, it seems that we did lose it until rc7.

Now, with rc8, this issue should be fixed. Would you mind checking the behaviour, please?

[vrtmrz]

P.S. We can make the configuration report with confidential things hidden by the Make report button.

Pressing this button will generate the following report. I hope we will find it useful.

``` ---- Obsidian info ---- Navigator: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) obsidian/1.4.16 Chrome/114.0.5735.289 Electron/25.8.1 Safari/537.36 FileSystem: insensitive ---- remote config ---- cluster: n: "1" cors: credentials: "true" headers: accept, authorization, content-type, origin, referer, ngrok-skip-browser-warning methods: GET, PUT, POST, HEAD, DELETE origins: app://obsidian.md,capacitor://localhost,http://localhost chttpd: bind_address: 0.0.0.0 enable_cors: "true" max_http_request_size: "4294967296" port: "5984" require_valid_user: "true" admins: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 vendor: name: The Apache Software Foundation feature_flags: partitioned||*: "true" chttpd_auth: hash_algorithms: sha256, sha require_valid_user: "true" secret: 24d06b7966c19d78f8f5653ffd2631da indexers: couch_mrview: "true" prometheus: additional_port: "false" bind_address: 127.0.0.1 port: "17986" httpd: WWW-Authenticate: Basic realm="couchdb" bind_address: 127.0.0.1 enable_cors: "true" port: "5986" smoosh: state_dir: ./data couch_httpd_auth: authentication_db: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 secret: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 authentication_redirect: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchdb_engines: couch: couch_bt_engine couchdb: database_dir: ./data file_compression: snappy max_document_size: "50000000" uuid: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 view_index_dir: ./data ---- Plug-in config --- version:0.23.23 remoteType: "" useCustomRequestHandler: false couchDB_URI: self-hosted(HTTPS) couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_PASSWORD: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 couchDB_DBNAME: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 liveSync: false syncOnSave: false syncOnStart: true savingDelay: 200 lessInformationInLog: false gcDelay: 0 versionUpFlash: "" minimumChunkSize: 20 longLineThreshold: 250 showVerboseLog: true suspendFileWatching: false trashInsteadDelete: true periodicReplication: true periodicReplicationInterval: 60 syncOnFileOpen: true encrypt: true passphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 usePathObfuscation: true doNotDeleteFolder: false resolveConflictsByNewerFile: false batchSave: true batchSaveMinimumDelay: 3 batchSaveMaximumDelay: 20 deviceAndVaultName: y-main usePluginSettings: false showOwnPlugins: false showStatusOnEditor: true showStatusOnStatusbar: true showOnlyIconsOnEditor: false usePluginSync: true autoSweepPlugins: false autoSweepPluginsPeriodic: false notifyPluginOrSettingUpdated: false checkIntegrityOnSave: false batch_size: 50 batches_limit: 50 useHistory: true disableRequestURI: true skipOlderFilesOnSync: true checkConflictOnlyOnOpen: false showMergeDialogOnlyOnActive: false syncInternalFiles: false syncInternalFilesBeforeReplication: false syncInternalFilesIgnorePatterns: \/node_modules\/, \/\.git\/, \/obsidian-livesync\/, ^\.git\/, \/workspace$, \/workspace.json$, \/workspace-mobile.json$ syncInternalFilesInterval: 60 additionalSuffixOfDatabaseName: 5647af6d6c53b9da ignoreVersionCheck: false lastReadUpdates: 23 deleteMetadataOfDeletedFiles: false syncIgnoreRegEx: "" syncOnlyRegEx: "" customChunkSize: 10 readChunksOnline: true watchInternalFileChanges: true automaticallyDeleteMetadataOfDeletedFiles: 0 disableMarkdownAutoMerge: false writeDocumentsIfConflicted: false useDynamicIterationCount: false syncAfterMerge: true configPassphraseStore: "" encryptedPassphrase: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 encryptedCouchDBConnection: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 permitEmptyPassphrase: false useIndexedDBAdapter: true useTimeouts: false writeLogToTheFile: false doNotPaceReplication: false hashCacheMaxCount: 300 hashCacheMaxAmount: 50 concurrencyOfReadChunksOnline: 30 minimumIntervalOfReadChunksOnline: 25 hashAlg: xxhash64 suspendParseReplicationResult: false doNotSuspendOnFetching: false useIgnoreFiles: false ignoreFiles: .gitignore syncOnEditorSave: false pluginSyncExtendedSetting: {} syncMaxSizeInMB: 50 settingSyncFile: "" writeCredentialsForSettingSync: false notifyAllSettingSyncFile: false isConfigured: true settingVersion: 10 enableCompression: false accessKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 bucket: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(9 letters) endpoint: self-hosted?((HTTPS)) region: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷(4 letters) secretKey: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷 useEden: false maxChunksInEden: 5 maxTotalLengthInEden: 1024 maxAgeInEden: 3 disableCheckingConfigMismatch: false displayLanguage: "" enableChunkSplitterV2: false disableWorkerForGeneratingChunks: false processSmallFilesInUIThread: false notifyThresholdOfRemoteStorageSize: 954 usePluginSyncV2: true usePluginEtc: false handleFilenameCaseSensitive: false doNotUseFixedRevisionForChunks: false showLongerLogInsideEditor: false sendChunksBulk: false sendChunksBulkMaxSize: 1 useSegmenter: true useAdvancedMode: true usePowerUserMode: true useEdgeCaseMode: true enableDebugTools: false preset: "" syncMode: PERIODIC dummy: 0 ```

(As you know, my setting is a bit dirty).

[foegra]

Very good, it works now. Good to see how You shape the plugin, with version 24 - it started to look better and more user friendly.