moneymanagerex / android-money-manager-ex

Manage your finances on-the-go, encrypted for security, and sync via your cloud
http://android.moneymanagerex.org/
GNU General Public License v3.0
484 stars 187 forks source link

Huge data loss because of malfunctioning/badly designed synchronization feature #1827

Open aardbol opened 1 week ago

aardbol commented 1 week ago

Describe the bug The app doesn't store records in the db that is defined in the Database path setting. The Synchronization button in the left menu has no effect to update that db with the latest records, hence keep increasing the gap of the live data and the data that is stored in that db. As a result, any backups made of that db will be useless and there's a huge risk of the Synchonization feature in the settings to cause an enormous data loss as it doesn't warn about a bigger db being overwritten by a smaller one or any similar security check.

To Reproduce

I often pressed the Synchronization button in the left menu, expecting that the original file (as shown in Database path setting) gets the latest updates written into it, that for a reason I don't understand aren't directly written to that db. I experienced this data gap issue with an older version before, so I decided from that point on to do manual syncs as explained. This worked in the beginning, hence why I didn't doubt its functionality later on.

Wanting to copy the latest version of the db to another device, I did the same thing again and copied the db, seeing that its history was more than 7 months old. So I Synchronized again, and saw that the size nor the last Modification Date changed for the db on the original device. So I went to Settings -> Synchronization, which is unfortunately very poorly documented in the app on what it actually does, and enabled Sync and without much thinking pressed the Download button, hoping that it would download the changes to the original db. Even though it should "Download... from the cloud storage", I never set up anything like that so I assumed it would cause no data loss. And without any warning about the huge data discrepancy, the app just overwrote the original db and removing more than 7 months of data...

Expected behavior There's a few issues here imo.

It's unclear why the original db doesn't get updated with the latest records. This should be basic behavior. It's also unclear why the Synchronize feature in the left menu doesn't perform that data sync

The Synchonization setting should clearly explain what's happening, and where it is downloading form and uploading to. There should be a warning about the risk of data loss, when downloading a db which doesn't have the latest live records. When enabling sync - in its current way of working -, it should upload the latest records immediately, and would've prevented Downloading an older db to overwrite 7 months of extra records as well. But generally, it shouldn't upload anything anywhere externally without a manual setup first, which also involves considering the security and privacy risks.

The app needs a proper backup feature. This flaw also shows again why syncing is not a backup. One issue or design flaw can cause a huge data loss. I have a backup process myself of my desktop computer, where the db gets synced to very often, but this is useless for the reason explained above: the db that always got synced, never got any updates from the app.

guanlisheng commented 1 week ago

hi @aardbol sorry to hear your data was lost and great you finally saved it with your own data backup.

in short, the app will temporarily store records in the db defined in the Database path setting as a cache, and keep it synced to Remote path periodically based on your synchronization setting (e.g. the interval to 5 minutes and ensure “Sync on start” from https://moneymanagerex.org/docs/features/usecloud/).

No idea why your sync did not work while can you share some information as the issue template suggested? Device Information:

aardbol commented 1 week ago

I'm afraid you misunderstood the backup part, due to the app not syncing the cache to the db by default or after me pressing the Synchronize button, the db in the backup also had +7 months of difference resulting in me having that huge data loss.

Fairphone 4 Android 13 Sync was disabled Latest app

guanlisheng commented 1 week ago

The synchronization should be enabled from the very beginning. along with upload and download, all these should have better hints.

may I have your cloud provider and specific version? we know DropBox application has some issue on its SAF support which caused MMEX can't fetch correct meta to perform the sync.

aardbol commented 5 days ago

Yes, I agree that the hints should be much clearer. But aside from that, it should have sane default settings to avoid such diasters, as well as have additional features for a perfect situation:

I can't stress enough how much this has inconvenienced me so I wanna help as much as I can to get this to work reliabily for everyone. I think I've used the app for at least 10 years and always have been happy, so I can also not thank you enough for continuing to maintain this app. If I had the skill and time, I'd even implement these features myself.

As for your question, I don't use a cloud storage for this due to the sensitivity of the data and never have set it up as mentioned earlier. I store the persistent db in a folder that's synced with many devices and some of them with an active automatic backup.

guanlisheng commented 5 days ago

the first two are in place and you can check them out there Main Menu -> Settings -> Synchronization. no matter cloud storage or phone storage, this should be always set up early. the last two would be improved a lot.

sorry and thank you again.

Leonardo @.***> 于2024年9月20日周五 03:21写道:

Yes, I agree that the hints should be much clearer. But aside from that, it should have sane default settings to avoid such diasters, as well as have additional features for a perfect situation:

  • enable cache to persistent syncing by default
  • offer automatic periodic sync but also a sync on app start setting
  • warn the user that an older version will overwrite a newer one on a manual download, so he's aware of the risk
  • add a proper backup feature that keeps multiple versions that's also configurable

I can't stress enough how much this has inconvenienced me so I wanna help as much as I can to get this to work reliabily for everyone. I think I've used the app for at least 10 years and always have been happy, so I can also not thank you enough for continuing to maintain this app. If I had the skill and time, I'd even implement these features myself.

As for your question, I don't use a cloud storage for this due to the sensitivity of the data and never have set it up as mentioned earlier. I store the persistent db in a folder that's synced with many devices and some of them with an active automatic backup.

— Reply to this email directly, view it on GitHub https://github.com/moneymanagerex/android-money-manager-ex/issues/1827#issuecomment-2362006011, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFQINM7TZOYWK7Q7DLTZCTZXMP3DAVCNFSM6AAAAABOH3IXLCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRSGAYDMMBRGE . You are receiving this because you commented.Message ID: @.*** com>

-- Lisheng

guanlisheng commented 5 days ago

as a quick response to avoid misconfiguration, the sync is always enabled, and sync service is always on after #1838 sync_setting