Specify custom folders for backup

[MuntashirAkon]

Some apps store important data outside the /Android/data/ folder. In such cases, AM should be able to back up the folders too. Note that there's no way AM knows in advance which data folders are used by an app. Therefore, the users have to specify them instead.

Due to the fact that each app uses different locations, this can't be added as a backup flag like others. Instead, the user has to specify the back up locations in the App Info tab. This could also be useful in future for other purposes.

Proposed workflow:

1. Go to App Info tab of an app
2. Click on the three-dots menu and choose Specify other directories
3. In the alert prompt, the user can insert a comma separated list of directories (which will also be displayed in the paths & directories section)
4. Now, select back up/restore either from the App Info tab or from the batch operations
5. Enable custom data flag
6. Click backup to make a backup of the folder(s)

[Cyberavater]

This is actually a nice idea! (Which I had but wasn't able to think/propose of a proper workflow like this one).

For this feature, I've got an extended suggestion;

Create a template in the GitHub issue regarding "Custom Data", this template would require the Package Name and Location of the custom data. For example, Package Name = "com.whatsapp" & Location = "/storage/emulated/0/WhatsApp" Then AM would have a default location for suggested/popular apps (Of course the user can remove it and add his own). Since there could be many apps that the dev might not want to install to verify the given location is correct/acceptable, the issue template would have the "help needed" tag and have a note like, "If you know the given location is correct, give an upvote in the OP". With 5 positive votes, the location will be added to AM's database and help more new users.

N.B. Storage Redirect follows somewhat similar methods to manage popular apps.

[MuntashirAkon]

Create a template in the GitHub issue regarding "Custom Data", this template would require the Package Name and Location of the custom data. For example, Package Name = "com.whatsapp" & Location = "/storage/emulated/0/WhatsApp"

This has some issues.

1. We have to take care of multiple users. So, there should be some sort of place holders for user ID such as /storage/emulated/%d/WhatsApp.
2. Google is making the SAF mandatory which means it's easy to know what folders are used by the apps by looking at the URI permissions granted to the app. At the same time, it became difficult actually figure out whether the SAF provider is a file provider or cloud provider. So, for these apps, we can't specifically provide any location (as they can be arbitrary set by the user). The only thing we can do is disabling backup of the files obtained via SAF for the apps. Therefore, we need another option, say enable_backup along with location and package_name. With SAF, there is another problem: determining the file location itself. As of today, I haven't yet figured out how to query providers using elevated permission. If this is possible, then we can provide maximum support for SAF. Otherwise, we can only provide support for the known and usable file providers such as (Android default) Files and MiXplorer.
3. There are a lot of apps and it won't be possible to package the custom locations of all apps in AM, besides we might want to enhance features in them such as adding custom scripts (such as #197) . So, it should be better if it is maintained by the community rather than myself as it would be a major burden for me. However, the issue is that we can't simply fetch anything from the Internet because of the privacy and security issues. I don't know if there's a solution to this problem either.
4. We might use the folders for additional purpose other than back up/restore or just simply displaying them in the App Info tab as I said in the first comment. One of the interesting feature would be to scan them DEX files as a malware (even the prominent ones such as Facebook) may attempt to load classes by downloading DEX files from the internet and keeping it there.

Then AM would have a default location for suggested/popular apps (Of course the user can remove it and add his own). Since there could be many apps that the dev might not want to install to verify the given location is correct/acceptable, the issue template would have the "help needed" tag and have a note like, "If you know the given location is correct, give an upvote in the OP". With 5 positive votes, the location will be added to AM's database and help more new users.

Voting is just as bad as democracy because not all people have the same knowledge or capabilities. It would better if the community members carries out an investigation before the approval.

N.B. Storage Redirect follows somewhat similar methods to manage popular apps.

Do they have any public repositories? That would certainly help.

[Deerp1976]

Storage Redirect is a paid app after using this for just 3 apps only. For most people this would be a problem.

Is it possible to have an option of custom folder for obb files? As these have to be in obb folder, having this as a default backup option would be very beneficial.

On Sat, 10 Apr 2021, 08:18 Muntashir Al-Islam, @.***> wrote:

Create a template in the GitHub issue regarding "Custom Data", this template would require the Package Name and Location of the custom data. For example, Package Name = "com.whatsapp" & Location = "/storage/emulated/0/WhatsApp"

This has some issues.

1. We have to take care of multiple users. So, there should be some sort of place holders for user ID such as /storage/emulated/%d /WhatsApp.
2. Google is making the SAF mandatory which means it's easy to know what folders are used by the apps by looking at the URI permissions granted to the app. At the same time, it became difficult actually figure out whether the SAF provider is a file provider or cloud provider. So, for these apps, we can't specifically provide any location (as they can be arbitrary set by the user). The only thing we can do is disabling backup of the files obtained via SAF for the apps. Therefore, we need another option, say enable_backup along with location and package_name. With SAF, there is another problem: determining the file location itself. As of today, I haven't yet figured out how to query providers using elevated permission. If this is possible, then we can provide maximum support for SAF. Otherwise, we can only provide support for the known and usable file providers such as (Android default) Files and MiXplorer.
3. There are a lot of apps and it won't be possible to package the custom locations of all apps in AM, besides we might want to enhance features in them such as adding custom scripts (such as #197 https://github.com/MuntashirAkon/AppManager/issues/197) . So, it should be better if it is maintained by the community rather than myself as it would be a major burden for me. However, the issue is that we can't simply fetch anything from the Internet because of the privacy and security issues. I don't know if there's a solution to this problem either.
4. We might use the folders for additional purpose other than back up/restore or just simply displaying them in the App Info tab as I said in the first comment. One of the interesting feature would be to scan them DEX files as a malware (even the prominent ones such as Facebook) may attempt to load classes by downloading DEX files from the internet and keeping it there.

Then AM would have a default location for suggested/popular apps (Of course the user can remove it and add his own). Since there could be many apps that the dev might not want to install to verify the given location is correct/acceptable, the issue template would have the "help needed" tag and have a note like, "If you know the given location is correct, give an upvote in the OP". With 5 positive votes, the location will be added to AM's database and help more new users.

Voting is just as bad as democracy because not all people have the same knowledge or capabilities. It would better if the community members carries out an investigation before the approval.

N.B. Storage Redirect follows somewhat similar methods to manage popular apps.

Do they have any public repositories? That would certainly help.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MuntashirAkon/AppManager/issues/360#issuecomment-817092627, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOYVKGVM4JEZY4QOG4ODPEDTH73SVANCNFSM42SNYVOQ .

[MuntashirAkon]

Is it possible to have an option of custom folder for obb files? As these have to be in obb folder, having this as a default backup option would be very beneficial.

There's already a flag in the back up/restore options for the OBB and Media files.

[Deerp1976]

My apologies I hadn't used the backup in some time and can now see the obb backup option. Thanks!

On Sat, 10 Apr 2021, 12:15 Muntashir Al-Islam, @.***> wrote:

Is it possible to have an option of custom folder for obb files? As these have to be in obb folder, having this as a default backup option would be very beneficial.

There's already a flag in the back up/restore options for the OBB and Media files.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MuntashirAkon/AppManager/issues/360#issuecomment-817119744, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOYVKGSEBR7N4GMM35TPVB3TIAXLRANCNFSM42SNYVOQ .

[Cyberavater]

However, the issue is that we can't simply fetch anything from the Internet because of the privacy and security issues.

Umm, maybe keep this suggested custom location disabled by default, then when the user enables it from the settings, give him a warning that the suggested location is shared/suggested by community memebers and might not be always correct, of course, we'd do our best to keep is as accurate as possible. If the user thinks there is a mistake he is welcomed to open an issue.

Do they have any public repositories? That would certainly help.

Yes, (https://github.com/RikkaApps/StorageRedirect-assets/issues). The app itself creates the json format, which is then shared via Github issues. It seems that the dev is mostly single-handedly verifying all the issues there, (for which many issues are still open). To takle this, I suggested help from other community members.

[Cyberavater]

We might use the folders for additional purposes other than back up/restore

Yes, we can even suggest/prompt the user to delete the folder when he is using AM to uninstall those apps, i.e when uninstalling Whatsapp, we can prompt him if he wants to delete the "/storage/emulated/0/WhatsApp" folder too.

[MuntashirAkon]

I think we need to spend more time thinking about its use-cases before start working on it. Features like these could have a large impact on the development if they were to change in future.

Let's see what we've gathered so far:

• Backup/restore. To be used in backup/restore under the external data flag. This could be useful for no-root users too since all they can backup are the external folders.
• App Info. To be displayed in the paths & directories section so that the user can open the folder using an external file manager.
• Clear Data. Delete the specified folder along with the systematic way of clearing app data.
• Clear Cache. Delete dot files within the apps (there should be a switch in its configuration to enable them). This is considered because many apps use dot files as a means to store cached files.
• Uninstall App. Uninstalling an app deletes the data folders too.

Let me know if other users find any more use-cases. The API will be designed according to these findings so that we won't have to deal with the compatibility issues next time.

[Cyberavater]

Backup/restore. To be used in backup/restore under the external data flag. This could be useful for no-root users too since all they can backup are the external folders.

I'd suggest keeping the custom folder separated from the external data since there is a chance that the provided custom location could be wrong, if anything comes unexpected from the restore, the user would have a good idea of what to try and unselect first.

You can/should be able to use the file from here (https://github.com/RikkaApps/StorageRedirect-assets) to automatically get updated custom folders for starters, since you already know he is a reputed dev.

[MuntashirAkon]

You can/should be able to use the file from here (https://github.com/RikkaApps/StorageRedirect-assets) to automatically get updated custom folders for starters, since you already know he is a reputed dev.

No, it's not that easy. The repository has no license, i.e. it's proprietary and I cannot use it.

[Cyberavater]

Is it because of legal reasons or ethical? I mean, should any legal loopholes be considered? Since that repo contains rules for 2.5k+ apps, collecting (and maintaining the future) rules for that many apps would be quite tedious.

P.S. It's just a friendly question out of concern for the future of this feature. This question doesn't intend to support/demote anything or imply that someone is holding certain views.

[MuntashirAkon]

Is it because of legal reasons or ethical? I mean, should any legal loopholes be considered?

Legal issues. From https://choosealicense.com/no-license:

When you make a creative work (which includes code), the work is under exclusive copyright by default. Unless you include a license that specifies otherwise, nobody else can copy, distribute, or modify your work without being at risk of take-downs, shake-downs, or litigation. Once the work has other contributors (each a copyright holder), “nobody” starts including you.

I mean, should any legal loopholes be considered?

You can ask the owners of the project. If they change the license or explicitly give us the permission to reuse their project under an open source license such as CC-BY-SA-4.0 and/or GPL-3-or-later.

[MuntashirAkon]

@Cyberavater: Also, the repository doesn't actually contain any rules. It only contains metadata. Rules are kept secret.

[Cyberavater]

The repo says it contains rules, https://github.com/RikkaApps/StorageRedirect-assets#readme I actually don't know any further.

But, you can check one of the accepted commits, https://github.com/RikkaApps/StorageRedirect-assets/commit/095208fb93c6722affc26e5c7ecd494b4ec7f943 If it is not what the issue is about then I'm sorry, I might've misunderstood.

[MuntashirAkon]

But, you can check one of the accepted commits, RikkaApps/StorageRedirect-assets@095208f

Yes, there are a few exceptions, but many of the rules are empty.

[MuntashirAkon]

Depends on #618

[MuntashirAkon]

I've given it a thought and came up with the following idea:

Use-cases

1. Backup/restore (separate flag: custom locations)
2. Clear data
3. Clear cache (subset of clear data)
4. Uninstall

The locations will be displayed in the App Info tab.

Format The locations could be anywhere. However, in order to make the restoring process easier across multiple users, it's necessary to specify some format:

1. Location types. There could be a few location format. For now, they are as follows:
   • Internal storage. Locations in /sdcard or /storage/emulated/{user} which will be replaced by the present internal storage during restore.
   • custom. Any readable and writable location within the device. This location will not be replaced.
2. Path. Path is relative directory to /sdcard (or the like) if internal storage is selected, otherwise it has to be absolute
3. Name. Each path must have a readable name.
4. Flags. The use-cases above are stored as flags.