Automatically encrypt exported files

[tb06]

Hello:

It would be very useful if the app had an encryption module. For an independent backup, a password could be entered when launching the backup. For a scheduled backup, a password could be stored in the scheduling settings. The password would be asked when restauring a backup.

With this functionality I would be confident in activating scheduled backup stored in my SD card and/or in a cloud service.

Thanks in advance!

[tmo1]

My instinct is to consider this to be out of scope for the app, and to tell you to use a transparent encryption layer (such as EncFs / gocryptfs / CryFS / VeraCrypt) + an app that speaks cloud (i.e. rclone), or just rclone with its crypt remote. Unfortunately, however, these encryption layers do not natively support Android. There are some third party apps that attempt to provide Android compatibility with these layers, but many are unmaintained / abandoned / of uncertain quality.

I am considering whether to go ahead and add native encryption support. In the meantime, you can try DroidFS or EDS (lite). There's also the Android rclone app RCX, which has a feature preview of exactly the functionality you want - but the documentation warns:

Feature Quality: Preview, data loss is expected!

so a more reliable solution is still needed ;)

[Ian2020]

If it's helpful Catima, another open source app encrypts exported zips with the help of the net.lingala.zip4j library:

https://github.com/CatimaLoyalty/Android/blob/e194cca5f17a3b62ab6d322f0c736f7a246e506b/app/src/main/java/protect/card_locker/importexport/CatimaExporter.java#L35C17-L35C27

[tmo1]

If it's helpful Catima, another open source app encrypts exported zips with the help of the net.lingala.zip4j library

Thank you! Zip4j does seem to be the way to add support for Zip encryption to Android apps, but at least at this point, I don't plan to take this route. I am reluctant to add any dependencies to SMS I/E than are not absolutely necessary, and I'd rather not rewrite all the Zip code to use Zip4j instead of the native Java / Kotlin functionality.

While taking a fresh look at this issue and the options mentioned in my previous comment, I came across Andrew Gunnerson (chenxiaolong)'s excellent looking RSAF project, which seems like it should provide a perfect solution to this issue, by allowing the use of rclone's crypt remote to transparently encrypt SMS I/E's output before writing it to local or cloud storage.

Andrew has actually been a very helpful contributor to SMS I/E, so I'm going to ping him for comment / advice about this proposed solution.

[chenxiaolong]

Hi there! Yep, RSAF should be a pretty reliable way to accomplish this.

• For cloud storage, you'd first create a remote for the cloud storage provider and then you'd create a crypt remote that points to the cloud remote. When SMS I/E (or any other app) writes to the crypt remote, it will encrypt the data before it gets uploaded to the cloud. (There's an option to block external app access to the cloud remote to prevent files from being accidentally uploaded unencrypted.)
• For local storage, you only have to create the crypt remote and then set its target to the local storage path (eg. /sdcard/SMSIE).

(However, saving to an SD card or USB storage will never work. This requires significant changes to rclone's code and RSAF uses rclone completely unmodified.)

Also, since RSAF uses unmodified rclone under the hood, even if I run out of time to maintain RSAF in the future, the backups will always be decryptable with rclone. The config file exported from RSAF can be directly imported into rclone.

---

That said, I would only suggest this for folks who are familiar with rclone already. RSAF isn't the most user-friendly app. It attempts to expose 100% of rclone's functionality as it is, without trying to make common features easier to use.

[tmo1]

Hi there! Yep, RSAF should be a pretty reliable way to accomplish this. ...

Thank you!

(However, saving to an SD card or USB storage will never work. This requires significant changes to rclone's code and RSAF uses rclone completely unmodified.)

Can you elaborate on the problem, or link to some documentation thereof?

That said, I would only suggest this for folks who are familiar with rclone already. RSAF isn't the most user-friendly app. It attempts to expose 100% of rclone's functionality as it is, without trying to make common features easier to use.

Noted. I use and really like rclone, although I'm no expert and I still need to check the documentation when setting up anything new, and I can certainly see that it might be daunting for those unfamiliar with it.

Nevertheless, at this point I'm going to officially recommend rclone / RSAF as the way to do automatic encrypted exports:

Zip encryption, in addition to the issues mentioned above, has a problematic history of insecurity and differing implementations, and I'm not sure of its current state.

I don't want to add native encryption, since rolling one's own crypto is always a bad idea, in addition to the fact that it would take work and add complexity and brittleness to the app. In general, I come from the *nix world, and subscribe to the Unix philosophy mantra of "Do One Thing and Do It Well", so I'd much rather leave crypto to other apps that have already implemented it and are good at it.

[chenxiaolong]

Can you elaborate on the problem, or link to some documentation thereof?

Sure. The problem is that in order to access SD cards or USB storage, Android requires that all access go through the SAF APIs, which conceptually works differently from plain old filesystem paths.

• To use SAF at all, rclone would need to be able to call into Java to access the SAF APIs.
• Accessing a single file that already exists is relatively simple. ContentResolver.openFileDescriptor() will return a regular file descriptor.
• Creating files is different. It's normally a single operation with filesystem paths, but it's two with SAF (create first, and then separately open). And the filename that actually gets created might not be what you requested.
• Accessing directories is way more complicated. There's no concept of paths, only URIs. For example, to access <SD card>/foo/bar.txt, it needs to start with the URI for the SD card, search for foo inside of it, and then search for bar.txt inside of foo. There is no direct access.

RSAF allows other apps to access rclone over these SAF APIs, but it does not allow rclone itself to access other storage over SAF APIs. If I had to guess, almost every line of code in rclone that accesses a file or directory would need to be changed to support SAF. There's no abstraction to make this easier because every OS rclone officially supports uses plain old filesystem paths.

---

Rant:

As one might guess, SAF's inability to access a nested file directly (need to search for each path component) is a big reason why SAF is slow. But in addition to that, the documented/built-in way of accessing a nested file via `DocumentFile.findFile()` is [very inefficient](https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:documentfile/documentfile/src/main/java/androidx/documentfile/provider/DocumentFile.java;l=332-348;drc=ff6e33e5915d8e253d1a6be79c7015ef61848522). Instead of doing a single query to list all child URIs + their filenames, it lists all the child URIs first and then separately queries the filename of each one. I've seen some devices where accessing `foo/bar.txt` took 10 seconds just because `foo` contained 200 unrelated files. With regular filesystem paths, this is something that should take milliseconds. At least most file managers are aware of `DocumentFile.findFile()`'s slowness and try to work around it.

[tmo1]

@chenxiaolong: Thank you for the detailed and clear explanation! So it's Android being Android again: making easy things hard and other things impossible :(

[tmo1]

Since I don't currently plan on implementing native encryption, I'm closing this issue, but feel free to continue the discussion here.