Allow app to sync data across devices

[iSoron]

Description

This feature allows the user to keep track of the same set of habits in two different devices (e.g. a phone and a tablet).

A preliminary version of this feature was released in Loop 2.0.0-alpha, but it was removed in later versions because it was not sufficiently polished. This issue tracks remaining tasks that need to be done before this feature is released to a wider audience.

Subtasks

• [x] Cryptography: Implement master key generation
• [x] Cryptography: Implement sub key derivation
• [x] Cryptography: Implement BIP 39 encoding and decoding
• [x] Server: Put data
• [x] Server: Get data
• [x] Client UI: Generate new code
• [x] Client UI: Enter existing code
• [x] Client UI: View code
• [x] Client UI: Disable sync
• [ ] Client: Encrypt database
• [ ] Client: Upload database
• [ ] Client: Fetch database
• [ ] Client: Fix habit deletion
• [ ] Server: Stop registrations if repository exceeds specified size
• [ ] Server: Enforce maximum upload size
• [ ] Server: Enforce valid keys
• [ ] Client: Notify user that sync has been disabled due to unknown error
• [ ] Client: Automatically re-upload data if decryption fails
• [ ] Client: Display last sync timestamp in settings
• [ ] Write docs/SYNC.md explaining sync protocol
• [ ] Update privacy policy
• [ ] Update main app description

[hobgoblina]

@iSoron Would it be possible to add a "set the data directory" setting so users could set up their own sync capability via services like Syncthing?

Could be a quick way to enable sync for some users while a native sync feature is still being developed.

[dessalines]

2nd @necropolina 's suggestion. I looked in the Android/data/org.isoron.uhabits folder, and didn't find any files other than explicit backups.... where is the data currently stored?

[fairyex]

@iSoron webdav might be a good solution.

[fairyex]

2nd @necropolina 's suggestion. I looked in the Android/data/org.isoron.uhabits folder, and didn't find any files other than explicit backups.... where is the data currently stored?

/data/data/, You cannot ascess it without root

[snoweuph]

@iSoron webdav might be a good solution.

Yea, using webdav would also make it nextcloud compatible, would love that.

[SkypLabs]

In the GitHub Discussion related to this topic, I suggested the use of Etebase: https://github.com/iSoron/uhabits/discussions/104#discussioncomment-2221110.

[lkraav]

2nd @necropolina 's suggestion. I looked in the Android/data/org.isoron.uhabits folder, and didn't find any files other than explicit backups.... where is the data currently stored?

/data/data/, You cannot ascess it without root

Also in https://github.com/iSoron/uhabits/discussions/104#discussioncomment-245999

This implementation would require Loop to store the database in a public folder. Besides the security and privacy implications of this decision, it should also be noted that Android in general is moving away from such practice. It's not clear if this solution would still work after Android Q.

Do you think @iSoron you could be persuaded to accept a PR to make database storage location a user preference?

Even though I have a rooted phone (Sailfish OS ftw) and can access /data/data/org.isoron.uhabits/databases/uhabits.db with root credentials, it's a bit of jumping through hoops, and many cannot do this at all.

This would unlock people using the app to make progress on their goals and issues via syncing the database file wherever they want via many mature 3rd party tools, while a more granular sync UX is under construction, with an unknown landing date (year even).

My use case: connect synced SQLite uhabits DB to an analytics tool like Metabase, to provide better daily parameter clarity to my health consultant to chase some health stuff. Isolating variables is a real challenge, and low-barrier tracking of inputs (habits) is something uHabits does really well :muscle:

[fillwithjoy1]

2nd @necropolina 's suggestion. I looked in the Android/data/org.isoron.uhabits folder, and didn't find any files other than explicit backups.... where is the data currently stored?

/data/data/, You cannot ascess it without root

Also in #104 (comment)

This implementation would require Loop to store the database in a public folder. Besides the security and privacy implications of this decision, it should also be noted that Android in general is moving away from such practice. It's not clear if this solution would still work after Android Q.

Do you think @iSoron you could be persuaded to accept a PR to make database storage location a user preference?

Even though I have a rooted phone (Sailfish OS ftw) and can access /data/data/org.isoron.uhabits/databases/uhabits.db with root credentials, it's a bit of jumping through hoops, and many cannot do this at all.

This would unlock people using the app to make progress on their goals and issues via syncing the database file wherever they want via many mature 3rd party tools, while a more granular sync UX is under construction, with an unknown landing date (year even).

My use case: connect synced SQLite uhabits DB to an analytics tool like Metabase, to provide better daily parameter clarity to my health consultant to chase some health stuff. Isolating variables is a real challenge, and low-barrier tracking of inputs (habits) is something uHabits does really well 💪

Currently I'm using "Save a Copy" by Rikka to save it to my downloads folder, but yeah the exporting data is a lot of hoops to jump through

[jruiz94]

I agree with @lkraav , this syncing would be super easily implemented by just letting the users set a storage location so they can then use Syncthing for example to synchronize the habits.

This is pretty similar to what apps like Aegis do. In the particular case of Aegis, it lets you set a folder where automatic backups of your 2FA codes wil be stored on every change you make. You can then later set that folder to be synced with Syncthing.