termux / termux-app

Termux - a terminal emulator application for Android OS extendible by variety of packages.
https://f-droid.org/en/packages/com.termux
Other
34.69k stars 3.65k forks source link

[Feature]: Enable Seedvault encrypted backups #2312

Open PandaCoderPL opened 2 years ago

PandaCoderPL commented 2 years ago

Feature description

In my opinion it would be good idea to let users backup Termux application using Seedvault. Here you can see example of how such backup option could be implemented in privacy-respecting way. More details about it here. Thank you in advance.

Additional information

I don't think so.

ghost commented 2 years ago
android:allowBackup="true"

has been disabled intentionally. Why: Android seem to unconditionally terminate application during backup. For people who use Termux just as a terminal, that's not a big deal. But for others, who run SSH tunnels, servers, etc, that's a nightmare.

Also it is unknown whether it will work properly. Termux often uses special files like symlinks (not saying about access modes) and overall data size often grows to gigabytes.

PandaCoderPL commented 2 years ago
android:allowBackup="true"

has been disabled intentionally. Why: Android seem to unconditionally terminate application during backup. For people who use Termux just as a terminal, that's not a big deal. But for others, who run SSH tunnels, servers, etc, that's a nightmare.

I understand your point. Would it be possible to let users change this option through settings or this is something that has to be set during compilation process?

Also it is unknown whether it will work properly. Termux often uses special files like symlinks (not saying about access modes) and overall data size often grows to gigabytes.

So it looks like it would be more complicated than I thought. Thank you for the quick reply.

agnostic-apollo commented 2 years ago

Related issue #122

Don't think you can enable/disable the starting of backup process which kills the app. The allowBackup value is stored in AndroidManifest which is built at compile time.

https://developer.android.com/guide/topics/data/autobackup#BackupSchedule

During Auto Backup, the system shuts down the app to make sure it is no longer writing to the file system. By default, the backup system ignores apps that are running in the foreground because users would notice their apps being shut down.

But defining custom files to be backed up seems to be possible with BackupAgent

https://developer.android.com/guide/topics/data/autobackup#ImplementingBackupAgent

https://developer.android.com/guide/topics/data/keyvaluebackup#BackupAgentHelper

And the max limit of 25MB is pretty low anyways.

https://developer.android.com/guide/topics/data/autobackup#BackupLocation

Backup data is stored in a private folder in the user's Google Drive account, limited to 25MB per app.

Caution: If the amount of data is over 25MB, the system calls onQuotaExceeded() and doesn't back up data to the cloud. The system periodically checks whether the amount of data later falls under the 25MB threshold and continues Auto Backup when it does.

https://developer.android.com/reference/android/app/backup/BackupAgent#onQuotaExceeded(long,%20long)

Seedvault seems to have 50MB

const val DEFAULT_QUOTA_FULL_BACKUP = (2 * (25 * 1024 * 1024)).toLong()

https://github.com/seedvault-app/seedvault/blob/11-2.2/app/src/main/java/com/stevesoltys/seedvault/transport/backup/FullBackup.kt#L32

PandaCoderPL commented 2 years ago

Related issue #122

I see.

Don't think you can enable/disable the starting of backup process which kills the app. The allowBackup value is stored in AndroidManifest which is built at compile time.

That's what I suspected unfortunately.

https://developer.android.com/guide/topics/data/autobackup#BackupSchedule

During Auto Backup, the system shuts down the app to make sure it is no longer writing to the file system. By default, the backup system ignores apps that are running in the foreground because users would notice their apps being shut down.

Thank you for the link.

But defining custom files to be backed up seems to be possible with BackupAgent

https://developer.android.com/guide/topics/data/autobackup#ImplementingBackupAgent

https://developer.android.com/guide/topics/data/keyvaluebackup#BackupAgentHelper

And the max limit of 25MB is pretty low anyways.

Yeah, that's really low. I think the only thing we could fit there would be Termux settings and maybe GPG/SSH keys because that shouldn't take too much space in most cases.

https://developer.android.com/guide/topics/data/autobackup#BackupLocation

Backup data is stored in a private folder in the user's Google Drive account, limited to 25MB per app.

Caution: If the amount of data is over 25MB, the system calls onQuotaExceeded() and doesn't back up data to the cloud. The system periodically checks whether the amount of data later falls under the 25MB threshold and continues Auto Backup when it does.

https://developer.android.com/reference/android/app/backup/BackupAgent#onQuotaExceeded(long,%20long)

Seedvault seems to have 50MB

const val DEFAULT_QUOTA_FULL_BACKUP = (2 * (25 * 1024 * 1024)).toLong()

https://github.com/seedvault-app/seedvault/blob/11-2.2/app/src/main/java/com/stevesoltys/seedvault/transport/backup/FullBackup.kt#L32

Interesting, so technically Seedvault developers could modify the code to allow backup of any size, am i right? Then Termux backup through Seedvault would make more sense if there would be some warning about Termux shutting down on every backup.

agnostic-apollo commented 2 years ago

Yeah, that's really low. I think the only thing we could fit there would be Termux settings and maybe GPG/SSH keys because that shouldn't take too much space in most cases.

Users would have to define a list of files they want to backup, cause $HOME often has libraries/modules or videos/photos and 25MB would hit instantly.

Interesting, so technically Seedvault developers could modify the code to allow backup of any size, am i right?

I haven't looked much at the code, but possibility yes.

Seedvault would make more sense if there would be some warning about Termux shutting down on every backup.

But termux-app can't enable the backup at all by setting allowBackup to true since it will get killed by android if users have enabled android's own auto backup. That will create additional problems for us in addition to termux processes being killed by OEM background restrictions/craziness.

Maybe a plugin app can be created just for backup since it will have access to termux files due to sharedUserId. I am not sure if only the plugin will get killed or termux-app as well by android. But I guess even if does, normal users who haven't installed the app would not be affected since allowBackup would still be false in termux-app. Additionally, it would also need to looked into if FileBackupHelper, etc would even allow backing up files from /data/data/com.termux instead of /data/data/com.termux.<plugin>.

Moreover, unlike other apps, termux provides lot of command line tools for customized backups for random files, including compressed and encrypted backups, that can be backed up locally as well as to different cloud storages or servers other than just google, like with rclone. So supporting auto backups is not as important, considering the issues.

https://wiki.termux.com/wiki/Backing_up_Termux

PandaCoderPL commented 2 years ago

Users would have to define a list of files they want to backup, cause $HOME often has libraries/modules or videos/photos and 25MB would hit instantly.

Maybe they could do so in ".backup" file? It would be pretty clear what the file is for. I think pictures and videos should be kept outside of Termux unless you actually need them inside terminal emulator for any reason.

I haven't looked much at the code, but possibility yes.

I mean you said that they did set it to 50 MB so I think they can set it to whatever else they want then.

But termux-app can't enable the backup at all by setting allowBackup to true since it will get killed by android if users have enabled android's own auto backup. That will create additional problems for us in addition to termux processes being killed by OEM background restrictions/craziness.

Good point.

Maybe a plugin app can be created just for backup since it will have access to termux files due to sharedUserId. I am not sure if only the plugin will get killed or termux-app as well by android. But I guess even if does, normal users who haven't installed the app would not be affected since allowBackup would still be false in termux-app. Additionally, it would also need to looked into if FileBackupHelper, etc would even allow backing up files from /data/data/com.termux instead of /data/data/com.termux.<plugin>.

I think only the plugin would get killed but in this case enabling backup would require installing plugin so users will be more aware of what they are doing and what may be the consequences.

agnostic-apollo commented 2 years ago

Well, someone will have to create a basic POC plugin (or modify something like termux-styling) and see how it behaves on different android versions.

Haiz14 commented 1 year ago

Since the discussion isn't going anywhere you should close it.