[Bug] microG doesn't appear on the app list

[m3thm4th]

microG Services Core doesn't appear on the app list, there are no active filters, all other apps (system and user) appear as normal

[MichaelZ4714]

What would you like that NeoBackup does with microG?

[m3thm4th]

@MichaelZ4714 back it up?

[MichaelZ4714]

I'm pretty sure you can't restore microG app so that it works. And I doubt that you can restore data. It isn't much and not difficult to configure. If you want to restore a previous release of the app it depends on the exact installation environment.

[m3thm4th]

Latest versions of microG save the WiFi and cell network database within the app. That alone is a very valid reason for wanting to back it up.

[machiav3lli]

Theoratically we may add a developer option for those with such brave hearts, but don't expect support, should the system break...

[hg42]

I never used microg or read a lpt about it.

which package name does it use?

I guess it's more than one package? as far as I remember it replaces google packages and uses their names.

On https://microg.org/ I see:

Components

Service Core (GmsCore) is a library app, providing the functionality required to run apps that use Google Play Services or Google Maps Android API (v2).

Services Framework Proxy (GsfProxy) is a small helper utility to allow apps developed for Google Cloud to Device Messaging (C2DM) to use the compatible Google Cloud Messaging service included with GmsCore.

Unified Network Location Provider (UnifiedNlp) is a library that provides Wi-Fi- and Cell-tower-based geolocation to applications that use Google’s network location provider. It is included in GmsCore but can also run independently on most Android systems.

Maps API (mapsv1) is a system library, providing the same functionality as now deprecated Google Maps API (v1).

Store (Phonesky) is a frontend application providing access to the Google Play Store to download and update applications. Development is in early stage and there is no usable application yet.

and in which package is the interesting data?

[hg42]

I always had in mind to read the exclusion patterns from a file, and that it could be overridden by another file in user space.

Preferably from a fixed directory name ("NeoBackup") in internal storage.

I didn't implement it, because a user space directory could be easily manipulated by other apps.

Today, that problem should be gone, as access to internal storage must be granted by the user (but don't forget root apps).

Such a directory could also be used to override other parts, like package.sh and tar.sh, and could also be used to trigger hook scripts, if they exist (e.g. to do something before and after a scheduled backup, like invoking rclone commands or rsync, or syslog etc.).

It would also be the place for user plugins. The simplest plugin would be a file list. With the override mechanism such a plugin could be provided internally by default, but also be replaced by the user.

MicroG could also be such a plugin, because it might be better, to restore single known files instead of the whole package data.

[MichaelZ4714]

I never used microg or read a lpt about it. which package name does it use?

com.google.android.gms

as far as I remember it replaces google packages and uses their names.

Yes, thats why Signature Spoofing is required

and in which package is the interesting data?

com.google.android.gms

There were installable addons for network location provider with previous versions of microG that could have held the data. But from v0.2.28 they are not supported anymore and a new api is not ready yet.

[hg42]

I made a test apk:

https://t.me/neo_backup/53944/56236

read the warnings in the second message

[hg42]

here the messages for documentation:

Harald, [3/14/24 1:09 AM]
-> #pumpkin_disclaimer

test apk for
github issue microG doesn't appear on the app list #811

the regular expressions used to filter are now loaded from
/data/user/0/com.machiav3lli.backup.hg42/files/assets/definition/

ignored_packages.regex filters the packages that do not appear in the package lists

do_not_stop.regex filters the processes(!) that should not be stopped

(apps usually run a process with their package name)

all the files in the assets folder can be overridden by files in

/sdcard/Android/data/com.machiav3lli.backup.hg42/files/

with the same sub-path, so
/data/user/0/com.machiav3lli.backup.hg42/files/assets/definition/xyz
is overridden by
/sdcard/Android/data/com.machiav3lli.backup.hg42/files/definition/xyz

(hmm, may be it should also be files/assets ? may be changed later)

Harald, [3/14/24 1:18 AM]
if you add the comment prefix to the gms line in that file, the microg package will be visible:
#| com\.(google\.)?android\.gms

this does not mean that backup/restore work for it.
That's your part 😊

There was some restructuring, and backup/restore was only tested with one simple package, so be prepared that this could fail miserably, and the gms package is also with high risk of failure.
Better use an emulator or similar for tests.

Harald, [3/14/24 1:25 AM]
btw. I had difficulties to edit the files in the Android folder.
Mixplorer always jumped into DocumentsUI (at Recents folder) wanting to get rights, but DocumentsUI cannot do that.
Xplore couldn't enter it either, and AppManager labs worked so far, but I couldn't start an editor from there.
At the end I worked over an adb connection, copying the file back and forth.

the /sdcard/Android/... folder is used in that apk, because the files must be used with java.File (at least the scripts). Other folders in /sdcard could only be used via RootFile or SAF... and access must first be granted (could be handled like the backup folder).

Not sure how to handle that in the following steps.

May be the files should only be imported to an internal location, where they can be used as a real file. This could also be faster (the package.sh and tar.sh are run on each package).

For now this is only a problem with the script folder, but plugins might also have scripts later.

Alternatively files could be read via SAF and piped into the shell.

[MichaelZ4714]

I made a test with the pumpkin in my WSA (Windows Subsystem for Android, Android 13). Yes, I got Signature Spoofing running in WSA. Root is through Magisk Delta. There was no problem to create the configuration files in Internal Storage Android/data/com.machiav3lli.backup.hg42. Using X-plore there was the DocumentsUI dialog to grant permission and then it was no problem to use that folder. The overlay was working correctly, I commented the line com.(google.)?android.gms in both ignored_packages.regex and do_not_stop.regex.

I made a backup of microG and then modified options for Location in microG settings. A restore was successful, but the revert of the changes was visible not before a Android reboot.

I then uninstalled the microG update 0.3.0 as user app and version 0.2.7 in Magisk module was active. I restored the NB backup. Again the version update was not seen before Android reboot.

I didn't do in-depth analysis. And there are no experiences with long-term stability after restore.

[hg42]

thanks for testing this.

You should probably disable such a package only in the ignore list, but it should still not be stopped.

I think gms is a basic service and stopping it could easily create a dead lock, even indirect, e.g. NB uses app xyz and xyz uses gms

[MichaelZ4714]

What happens when a system app is updated? Isn't it also stopped?

[hg42]

[I didn't read your comment correctly, so the following is about NB stopping processes]

if the process runs as a system user (<10000), it is not stopped.

As far as I remember, processes that have open files in the app directories are not filtered again for system users. That would probably mire or less disable that strategy.

Adfitionally, services for other apps could eventually run as app user.

Well, I never investigated such things for each possible combination, that would be way to much.

I tried to find rules that work. But I am not very convinced. At least, the open files thing is more or less ineffective.

[hg42]

What happens when a system app is updated? Isn't it also stopped?

I don't know... it may be that the system plays some tricks.

Anyways, the problem isn't the stopping itself, but which processes depend on those stopped processes.

Apps like NB use official APIs. And gms is for user apps.

The system can use internal APIs or call functions directly. It does not depend on gms.

[machiav3lli]

Thanks @hg42

[machiav3lli]

What happens when a system app is updated? Isn't it also stopped?

If it's running it gets relaunched/restarted (signal INT/STOP I would guess), so not "killed" as in KILL/TERM.

[mvglasow]

What happens when a system app is updated? Isn't it also stopped?

I don't know... it may be that the system plays some tricks.

How does NeoBackup restore system app APKs – does it go through the package manager, or does it copy the APK to the exact paths it was backed up from?

In the latter case, a dirty hack might be to first copy the APK, then install the same APK again through the package manager (which might need some extra flags to keep the package manager from balking at installing the already-installed version). That should trigger whatever the system normally does when a system app is updated. Not sure what happens then, but it would have to be tested.

I see the following options after restoring an app which matches do_not_stop.regex:

1. Reboot (the Windows way – annoying but relatively foolproof)
2. Install the APK a second time using pm install or the corresponding API call, with flags to ignore the version as needed (if that works, it’s probably the best tradeoff between stability and minimizing user annoyance)
3. Stop the app (may break things, depending on system configuration)
4. Do nothing (restored app may not work and/or apps depending on it may misbehave until the system is rebooted)

This could be implemented by displaying a message after restoring one of the apps in question, informing the user that some of the restored apps may require extra steps to work correctly, and let the user choose which steps to take. The options could be configurable, with only 1 and 4 available by default, and the other two available only after enabling some kind of ”expert mode” in settings.

[hg42]

I am quite sure, system apps cannot be handled with a general strategy. At least some of them are system apps, because they need to be. Well, I never proved all this, but it's kind of obvious. If it's not like that currently, it can become like that in the future.

Just take a look at why they are system apps:

• they are delivered with the ROM
• they are stored on the system partition (uninstalling them does not free user space, that's why they are only disabled, well usually)
• they can have special permissions (including some that are not available for user apps)
• there might be scripts that handle updating and other situations (all the states an app can have)
• I read somewhere that Play Store (or the installer or similar) knows something about them and has special handling for some of them
• user, group, SELinux contexts, etc. and their handling are usually very different from other apps
• they are arranged in sub-systems with interdependencies ... probably more ...

Some system apps are like other apps, they could live in user space, but happen to be delivered with the ROM. I think, these are the only ones that could be handled generally. But I guess it's not worth the hassle. Many of them are seen as bloatware (e.g. facebook installed as system app).

Personally, I would not try to build special handling into NB.

This would be a typical task for plugins.

Some knowledgeable users of those apps need to write those plugins, because they need to be familiar with the internals and specialties of the particular "app".

Which also means, the plugin interface must be flexible enough, even for future changes in NB and Android.