no planned backup of special backups

[DJaeger]

I have configured a daily planned backup of all apps. Today I have recognized, that there doesn't exist backups of the the special backups.

[jensstein]

do you mean that there isn't an option to specify only doing the special backups (that is, are you requesting this as a feature)? or do you mean that the special backups are not included when you schedule a backup of all apps?

[DJaeger]

I meaned the latter. But currently I'm unsure, as I retried it after fresh install and it seems there are also others left out as it seems to stop somewhere before ready. Maybe timeout of background task, as it makes backups of 216 apps (with encryption)?

But a selection for planned backup of only special backups would also be great. ;-)

[DJaeger]

I try it again and let the screen on manually (by moving around on the screen sometimes).

[jensstein]

ah, then i think it may be that it dies because it holds a wakelock for too long. i am aware of that issue. it could be something else though. please send me an email with a logcat in any case. it's also interesting if your idea with keeping the screen on makes a difference.

[DJaeger]

I have not seen something relevant last time but will recheck this time.

[DJaeger]

Mhm, have now seen, that it takes very long on backup of Radiobeacon, as it contains a database of ~0,5 GB and a full country offline map of 1,35 GB in its data directory...

[jensstein]

yes, very large amounts of data are another issue to handle. i am planning to include a blacklist which could at least mitigate problems with uncommonly large amounts of data. still, let it run for a while and try to get me a logcat.

[DJaeger]

I/ActivityManager( 754): Killing 26434:dk.jens.backup/u0a20 (adj 9): excessive cpu 499640 during 900038 I/WindowState( 754): WIN DEATH: Window{42cf20f8 u0 dk.jens.backup/dk.jens.backup.Scheduler} W/PackageParser(23480): Unable to read AndroidManifest.xml of /storage/sdcard1/backup/Oandbackup/org.openbmap/external_files/org.openbmap.zip I/ActivityManager( 754): Start proc dk.jens.backup for activity dk.jens.backup/.Scheduler: pid=6472 uid=10020 gids={50020, 1028, 1015} D/ActivityThread( 6472): handleBindApplication:dk.jens.backup I/oandbackup( 6472): openpgp-api service bound I/ActivityManager( 754): Displayed dk.jens.backup/.Scheduler: +513ms I/Timeline( 754): Timeline: Activity_windows_visible id: ActivityRecord{429776f8 u0 dk.jens.backup/.Scheduler t11} time:56070098

[DJaeger]

As the notification remains on crashing background task, you don't recognize, that it crashed

[ildar]

@jensstein, you may consider precalculating DATA size and make the app skip those apps that larger then predefined

[DJaeger]

@ildar: And don't let users backup big apps? I don't think this is a good solution.

How about a blacklist? Skip for example:

• thumpnail folder
• cache folder
• *.map files
• tiles folder
• tmp folder
• temp folder

and so on?

[ildar]

Isn't cache skipped already? 😱

[DJaeger]

I'm not sure, but I think yes. But I don't wanted to leave it out of this list, to be more complete. I think the way of doing it for cache, if its excluded already, could also be used for the others.

Also there should be a notice to the user before, that "by the apps restorable not user specific data", is not included to save space.

[jensstein]

no, cache isn't skipped (at least not intentionally - if it isn't included in your backups, please report it as a bug). i was originally thinking of a blacklist for app names (package names) but a blacklist for path names sounds like a good idea. it would require the copying to be done with the individual files for sources and not on the whole directory as it is now (with cp -r). but that may not be a problem. in the meantime you can set a custom list of package names to be backed up by your schedule from which you could then exclude problematic apps.

as for the program dying, it happens because android (ActivityManagerService) judges that it has run for too long - in this case used to much cpu time. so the process is killed to free ressources to the system but the app doesn't crash with a java exception so it can't be caught and handled like normal errors. i am considering whether a workaround would be to split a given operation into segments and then re-acquire a wakelock and such for every segment.

[DJaeger]

It could be done with find and cpio

[DJaeger]

The issue with dying is, that you can not realize this, except you read the log or running processes, as the notification of the backup progress is still shown. Did I understand it correct from other apps, that an app will not get killed, when showing a sticky notification. So why not simply change backup progress notification to sticky?

[jensstein]

i'm not sure find would be the best option here. it would be necessary for the busybox installation to a find command that either understands ! or has the -not flag implemented to ignore files. i may well be that every busybox has this, but i've had bug reports before because of missing implementations (very minimal busybox installations) so i would rather not rely on it too much. i think i'll try doing it with a for loop in the shell or something like that at first. i'm not sure i can see how cpio will be helpful since we're already archiving in zip format.

and yes, a service running in the foreground while displaying a notification might be a solution here.

[DJaeger]

I have find and cpio available. You said you use cp -r and the common alternative for this, which is aware of exclustions, is a combination of find and cpio.

[jensstein]

yes, but the point is that they and the needed flags may not be available in every busybox installation. i still don't get the use of cpio. it's a program for archiving data not for copying data in general (like cp). i know you can use it with find but it wouldn't really be appropriate here, would it? (if i were to use find i would probably use the -exec flag)

[DJaeger]

You mean something like this?: http://stackoverflow.com/a/4586025

Some other ideas: http://stackoverflow.com/a/12968671

http://www.linuxquestions.org/questions/linux-newbie-8/how-to-do-recursive-file-copy-of-directory-for-specific-files-199134/#post1018622

http://superuser.com/questions/299938/how-can-i-recursively-copy-files-by-file-extension-preserving-directory-structu

[jensstein]

yes, the first link is how i would use find if it ends with that. (also, from the cpio man page i have just noticed that it has a "copy-pass mode" which seems to make it copy files without creating an archive. i would still prefer cp though because that mode might not be available in a given busybox installation, so there would have to be a fallback in any case.) thanks a lot for these suggestions :)

[fajabird]

Please consider to reanimate you idea of a per-app blacklist, so it would become possible to backup basically e.g. all user apps (especially the ones installed after the backup job has been created) excluding those in the blacklist, e.g. apps that don't need a backup because of some reason (too many/large files that are not critical, like Google-Play-Music, or apps which break the backup process for other reasons). This would really be a uniqe feature (maybe titanium has this too, don't know).

I suggest to implement an additional button "Blacklist" on the scheduler page. This way it will work independently of the app-list and job chosen (all vs. user vs. system apps etc.)

[jensstein]

the scheduler now has a blacklist :) at the moment it is global so that it affects all schedules - blacklists for individual schedules might come in the future. it is configurable via the option "set global blacklist" in the scheduler activity. please try it out and report errors: https://drive.google.com/file/d/0B0bNUNWd2wPPTjJ3T25KaEVQd2c/view?usp=sharing :)

[fajabird]

Hi,

what great news. I allready downloaded it and started my frist blacklist. While I can't tell if its working at that time I allready have some minor issues:

1. Don't understand the sorting. I usually use app names for sorting (not package names) - ideall the blacklist should use the same settings.

2. It would be very handy to have all checked items at the top of the blacklist.

... to be continued :-)

Regards Artur

the scheduler now has a blacklist :) at the moment it is global so that it affects all schedules - blacklists for individual schedules might come in the future. it is configurable via the option "set global blacklist" in the scheduler activity. please try it out and report errors: https://drive.google.com/file/d/0B0bNUNWd2wPPTjJ3T25KaEVQd2c/view?usp=sharing :)

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/jensstein/oandbackup/issues/100#issuecomment-293043195

[jensstein]

@fajabird thanks for the suggestions, the list is now sorted by label since that is what's shown (it was sorted by package name before). and checked items are now at the top of the list. :)

[fajabird]

Hi,

blacklist works very well so far. Where can I download this build?

Regards Artur

Am 12. April 2017 21:56:48 MESZ schrieb jensstein notifications@github.com:

@fajabird thanks for the suggestions, the list is now sorted by label since that is what's shown (it was sorted by package name before). and checked items are now at the top of the list. :)

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/jensstein/oandbackup/issues/100#issuecomment-293689749

[jensstein]

the same link now has an updated package: https://drive.google.com/open?id=0B0bNUNWd2wPPTjJ3T25KaEVQd2c :)