"Work on internal cache only" keeps reverting back

[ustulation]

Version: 1.10-pre

There is an option for Work on internal cache only which makes keepass2android read/write only to whatever cache/db it's using (or so is my guess). However if I then hit Synchronize cache copy with the source it does a 2-way-sync such that the cache and the source-db both get updated with each other's contents, but then it reverts back to using that source-db again and I need to press the Work on internal cache only again in order to keep that going. Is this intended or is this a bug?

---

The reason I ask is because to me this is a very useful feature to not have data-loss when sync-ing multiple devices. This would make it behave like a "master-local" sync. Say if it never reverted (unless explicitly told so via some other menu item) and continued working on the internal cache, then the source-db on the filesystem can just be overwritten during my-dumb-file-sync-mechanism. It wouldn't matter even if keepass2android was offline at the moment (say because background app got killed etc). When it starts up again, it just uses it's local copy (the cache) and all its previous changes not sync-ed with anyone else yet are still preserved. Now I can just hit Synchronize cache copy with the source for a one-time 2-way-sync which will update both the local copy (the "cache") and the source db (master). The source db can then be sync-ed with whatever mechanism one is using with say keepassXC on desktop etc. We would never have to resolve conflicts due to the dumb-file-sync, because keepass* family of programs already merge dbs flawlessly and in case of true conflict (same record updated differently) just keep the one with the latest timestamp and put the others in the version history (which can be restored from if needed).

Currently to achieve this master-local sync, I need to remember to press the Work on internal cache only again immediately after Synchronize cache copy with the source otherwise it loses that status.

[PhilippC]

KP2A's default behavior (unless you enable "Work on internal cache" explicitly) is to synchronize on each read/write operation. Why do you need to worry about being offline or the master file having been modified? These cases are handled gracefully without enabling the option all the time?

[ustulation]

I tried to explain it above but let me try again. The TL;DR is that in my understanding, no, the cases aren't handled gracefully in any non trivial set up. I actually tested this and it has a potential for data loss unless you have more involved setup than necessary.

Details: Consider this: there's keepassxc (KXC) on laptop and keepass2android (K2A) on mobile. We want the db to be non-internet facing (ie should never leave machines controlled by solely us) so we use something like syncthing to do p2p sync over local lan.

Scenario without internal cache option: k2a db and kxc dbs are both identical to start with. Let's say it's db-X. Changes are made at both places. K2A adds new entry A and KXC adds new entry B. The lan or the sync tool is down so at this pount the db's differ. Android kills K2A for some reason (or you restart your phone or whatever). You haven't logged into K2A yet. The lan/sync comes back on. KXC db has a more recent timestamp (assume this) than K2A db timestamp. So the sync renames/copies K2A original to some conflict filename DB-Conflict and then overwrites DB-X. So once again DB-X has same content on both machines but with only KXC's changes. K2A's changes are in DB-Conflict.

We want the sync to be dumb and agnostic since keepass family of apps know how to merge and deal with conflicts themselves. So assume we forget about DB-Conflict (just delete it or ask sync tool to not even generate it in the 1st place).

K2A starts, reads DB-X, and has now lost its changes forever.

If you always had it working on internal cache, nothing would be lost since no one is touching the "local" db (internal cache). When you want to sync with source explicitly in the K2A app, it'll realise the divergence and simply give you an option to merge changes, after which the master DB-X will contain both KXC's changes (which it already had) and K2A's changes due to the recent merge.

This is called "master-local" sync and is very powerful achieving eventual consistency among potentially 10s of peers. The master (DB-X) is sync'd and travels over the network while the local (K2A's internal cache and in case of KXC its loacl db of which DB-X was a "keeshare") is never touched by anyone except the application.

Currently i need to remember to press work on internal cache only after doing the sync with source db option in K2A.

1) if it remembered this and worked only with internal cache even after the occasional and manual sync with source, it would be awesome and not error prone because the user forgot to tap on work with internal cache only option.

2) it would be even more awesome if it behaved like KXC where the app works with its local copy while automatically monitoring the master copy. That way we won't even need to hit sync with source since that would automatically happen each time it detects there's a change that is either in internal cache (the local) but not in master (DB-X) or vice versa. This is how KXC works on desktops and is awesome.

[ustulation]

(Sorry i am n mobile and accidentally hit close while typing out a response to you @PhilippC and i have reopened it)

[PhilippC]

thanks, I understand your point now. I think a setup with something like a local SFTP server on the laptop would be easier (and works nicely with the current implementation), but it's a valid feature request to have an option to "stay on internal cache"

[ustulation]

Cheers, i don't think sftp or anything would make a diff here, rather make it worse in another way: now it's no longer decentralized and there's a single point of failure (needs to be reachable all the time etc), whereas in p2p every machine is like a server. But even if we set that aside, the problem is when K2A goes offline (app is killed).

As long as the app is running there's no problem with any approach since its "local" is its stuff in memory. If the stuff in disk changes it can easily re-sync since nothing touches its memory. However once it's killed then you can get into all sorts of edge cases. In that case if it always worked with internal cache, then that cache becomes the "local" (along with the in-mem data when the app is running, and exclusively just by itself when the app is not).

---

Anyway since you approve of the feature request, it's all good. Also see if you see the merit of point (2) too where it monitors the "master" (source db) and automatically does a two way merge when it detects divergence. That way we don't even have to manually do syncs every now and then (btw this is how KXC with its keeshare works, so there's a precedence and won't be a new concept, which is a good thing IMO)

[PhilippC]

KP2A has a local cache copy (which is read from/written to when being offline/out-of-LAN) and synchronizes with the master through SFTP nicely, merging files when necessary. This of course assumes you use KP2A's built-in SFTP implementation instead of third-party apps. The only issue I see is if your "dumb file sync" cannot handle a conflict and creates a "conflicted" copy.

[ustulation]

Ah so there's a built-in sftp impl in KP2A that works by auto-merging instead of overwriting local cache. That's nice to know, cheers! I just use vanilla KP2A and that, when it's killed and source db changed and then started again, simply says it updated the local cache with source db, basically overwriting and thus losing all the changes which were exclusively in local cache not in source db ("master"). I faced this data-loss in the past. So maybe the SFTP impl is different as you say and it instead tries to 2-way-merge the local-cache and source-db when it restarts and detects divergence.

Anyway, for my use case I won't be able to use any server based sync mech since (1) I have many machines (and even on a single phone, many android profiles) so I need p2p sync for which things like syncthing just work out of the box (and not just for kdbx files) and (2) I doubt if all keepass derivatives have such built-in SFTP mech natively (for eg. KeepassXC and so on). So 3rd party sync keeps things nicely detached and since that doesn't know about kdbx etc. it's just a dumb sync expecting the apps to resolve stuff themselves, which they all do including KP2A just that in case of KP2A it's a series of button clicks as described earlier.

I had made a 4 part youtube vid sometime back showing how all of this works in practice but I'm no youtuber so it's probably technical and boring for most people :) It's also one of the reasons why I prefer K2PA over KeepassDX on android.