Support Marshmallow (6.0)

[mfonville]

This bug will be the central tracking bug for marshmallow support.

• [x] The website will have to be updated (initially greyed out, but JavaScript can already be added)
• [x] Makefile has to be updated
• [x] build_gapps.sh needs to be updated
• [x] apk lib extraction has to be skipped for 23+
• [x] 6.0 packages need to be added to the repos
• [x] buildbot has to be updated (will be done when aosp rom will be available)

[bgiesing]

From TKs thread, do we need to re-evaluate for M?

---

[quote name="Spasticdroid" post=62385523]A little OT here, but having a look inside the image of Marshmallow Preview 3, it seems (for just about all Google apps) that you won't have to pull out the libraries and put them in their own folders anymore, which saves quite a bit of space, especially with Chrome and WebView :victory:[/QUOTE]

Yep. This will also make it easy peasy for anyone to integrate an update from the Play Store right back into their system partition. The value in having an updated GApps package will be greatly diminished if things stay this way through the release. Anyone can sideload an update and integrate right into their system partition without having to wait for an updated GApps package.

I'm extremely happy to see this change.

[mfonville]

We can pursue such a project, to move Google apps from data to system. But after some consideration, I doubt it will be the holy grail. Since not all gapps are aways available from the play store eligible for installation on all phones (e.g. Google keyboard). Also installing all apps on 1 go after installation of ROM is convenient. And when updating the ROM, all apps (and their data) would be wiped, unless some addon.d hack would be applied. Also, symlinks to systemwide libs, that really emulate the Nexus filesystem layout, would not be easily possible.

So I think the 'regular' packages will still be useful on 6.0. The libraries change is trivial, and will make things more simple. It needs more study for how multi-arch apks will be handled.

[bgiesing]

@mfonville What about this init.d script by @osm0sis. We could include it in the zip?

[mfonville]

We could do something like that. But it would need much more work. Like I said earlier, for addon.d stuff. But that script e.g. does not support multi-arch, and would also greatly fail when installing/updating Chrome (and its crazy libs). But focus should be to first get regular gapps working. After that we can study this.

[mfonville]

I studied now the current Marshmallow preview. But apparently not all APKs have their libs fully embedded. The symlinks from e.g. lib/arm/*.so to /system/lib/ still exist for Chrome, Google Keyboard etc. So it looks like Google has created a Frankenstein hybrid :-/

[mfonville]

If the libs don't have to be pre-extracted btw, we can could include different versions of each APK in the package. @rapperskull what is your opinion about this? Should we only include the latest version, irrespective of the DPI, or each DPI that we have available even if the version is older, or some kind of hybrid, where this choice is made per application? This issue/choice is btw related to #97

[rapperskull]

As for as the Marshmallow changes, let's wait for the final release to see if there will be other changes. Apps like WebView probably need to have symlinked libs, so that they can be accessed system-wide, so we should make a list of what apps have their libraries extracted and what haven't to correctly build our GApps. As for as the DPI question, we have to keep in mind that it's just a temporary problem, since it only occurs when the apps get first updated and solves itself when we obtain all the APKs. We can think about including old replacing versions, but we have to find a way to implement the change. Google could for example completely remove a DPI variation and we would still install an old version as a placeholder. I think that the best is to always include only the latest version (that's why the users always flash the latest package), but there are points to support the other view too.

[TheCrazyLex]

Hey, I can help you to create some ROM side tweaks :)

I have a commit ready to auto grant needed permissions for core google apps already.

Maybe I can help you you on the prebuilt library thing too ^^

Alex

[rapperskull]

Hi, feel free to make a pull request with the changes. Contributors are always welcome.

[TheCrazyLex]

Hi, @rapperskull but my patch is on ROM level, it applies on frameworks_base.

[rapperskull]

OK, then you'll have to share it with ROM devs. We are currently looking for a way to grant the permissions without any change from the ROM side, but if this patch will become mainstream all ROMs will support it and everything will be easier.

[TheCrazyLex]

Maybe if you won't find a way, you can create that repository for reference to ROM devs, so they can directly cherry pick from there?

Also it would be cool if you could share the list of apps to which I should grant permissions ^^

Alex

[rapperskull]

All apps from the core package need to have granted permissions by default, and also all the apps that cannot be installed from the Play Store, like Google Play Store itself, Google Calendar Sync, Face Unlock, Google Exchange Services, Android for Work, GCS, Google Play Services, Google NFC Tags, Sound Search (since the Play Store version is outdated), Google Webview and DMAgent.

[mfonville]

@TheCrazyLex Thanks for your help. @rapperskull he did respond to the issue because of the blogpost I wrote: https://plus.google.com/+OpengappsOrg/posts/CmfvzqwQZM7

So the best would be a double approach, both a script in our package, trying to fix the permissions. But fixing it ROM side first, and making that patch popular, is even more important, because then we'd have a reference for what the script should achieve, and we would be able to test.

The full list of apps that need the permission are those of the list of gappscore that can be found in inc.buildtarget.sh and inc.compatibility.sh

[TheCrazyLex]

@rapperskull @mfonville Ok, thank you, very good list :+1:

And I should really grant ALL permissions to these apps?

[TheCrazyLex]

Also I don't really understand the logic of the issue with the prebuilt libraries... I would like to fix, but need to understand first.

[deadman96385]

@TheCrazyLex I was wondering if you could upload your changes to enable perms on google apps by default i am trying to make a basic gapps package for an x86 device so i can start getting sdk 23 app versions for open gapps and other users

[TheCrazyLex]

@deadman96385 Yes, I am going to upload the patch to the Exodus Gerrit when ready (during the next 2 days) I will also need people to test,cause I am not sure if my mechanics can auto allow in time...

Will update you here :)

Alex

[TheCrazyLex]

@mfonville

[TheCrazyLex]

Ok, guys. I am now at a point where I can start adding all the apps and specifying the permissions.

However I have no 6.0 device here... And on 5.1 the apps need nearly every permission.

Maybe someone with 6.0 can look at all the core gapps (also hidden ones) and tell me which permissions they really need on 6.0

I need your help.

Thanks :)

Alex

[TheCrazyLex]

Ok, if you'll need it. here is my document where you can see which permissions are required in 6.0 by which apps

https://docs.google.com/document/d/1OR8c1PkVjHT8OEXZvQ92qnOKMVwKoHz4fzB2wqEqVbk/edit?usp=sharing

Alex

[TheCrazyLex]

Ok, my change is ready and online.

@mfonville @rapperskull please review and test it :)

http://exodus-developers.net:8000/#/c/1974/1

Alex

[rapperskull]

I'm not a Java expert, but it looks good. I'm currently not able to test. The only doubt is about the permissions. Are you sure you included all of them? How have you compiled the list?

[TheCrazyLex]

@rapperskull I am pretty sure the permissions are right. What I did is, I found someone from the Exodus community who has a nexus device with a stock google 6.0 image installed and handed him the list.

I asked him to find every app from the list and send me a Screenshot of the permissions used.

Then I wrote them into the list and... Then wrote the code using Google's own autogrant system.

I cannot test myself though as my phone is still not booting with 6.0..

If something is missing a tester would see it in the log and post it here :)

Alex

[mfonville]

@TheCrazyLex thanks for your work. Unfortunately I have no 6.0 device myself, so we are dependent on when @rapperskull will have time to do so.

[deadman96385]

I have a work in 6.0 Rom just no 23 sdk apps for it because it being x86

On Sun, Nov 1, 2015, 11:55 AM mfonville notifications@github.com wrote:

@TheCrazyLex https://github.com/TheCrazyLex thanks for your work. Unfortunately I have no 6.0 device myself, so we are dependent on when @rapperskull https://github.com/rapperskull will have time to do so.

— Reply to this email directly or view it on GitHub https://github.com/opengapps/opengapps/issues/93#issuecomment-152848819.

[TheCrazyLex]

@mfonville maybe I can help to resolve the other 6.0 issues too? Just tell me what to do :)

[cryptomilk]

Only granting permission by the package name opens a security hole. You could install an app with the same name and get the same privileges as the gapps!

You should also check the signing key of the google app!

[cryptomilk]

However, how is this done on a stock ROM? Did someone check the framework code?

[TheCrazyLex]

@gladiac well I've done it in the style Google does it with the stock phone app etc.

Also it is not a security issue as Android for example automatically grants preset permissions to any browser. So if my patch is a security issue then Android has a big security issue too.

Also the user decides by himself what he installs from unknown sources and what he doesn't. Play Store is 100% safe in this case

I only grant critical permissions to Google play services, every Gapps package has that included.

Alex

[TheCrazyLex]

And all these packages are core packages, means included in every Gapps install

[mfonville]

@TheCrazyLex One of the other "big" issues that is still open, is that Google's ROMs apparently accept APKs where the libs are still inside the apk, and they are loaded/extracted on-demand. While the AOSP ROMs still expect the libs to be pre-extracted. If you could look into that issue? What causes the difference in behaviour? If that would be fixed, we would be good to go!

[TheCrazyLex]

@mfonville I will look into it :)

You are saying that it was working in 5.1.

I mean this extracting on-demand

Alex

[mfonville]

No, in 5.1 it wasn't working, 5.1 needed the pre-extracted libs. But that was on both aosp and Google ROMs the same. But now, Google ROMs seem to be able to do it on-demand, but aosp not. So Google is doing a trick that ain't in aosp (enabled?) (Yet?)

[TheCrazyLex]

@mfonville it is normal, that Google doesn't publish all code....

Most probably we have no chance :( I will however recheck

Also I wouldn't say it's a deal breaker/blocker issue as you had to pre-extract libs in 5.1 too.

What do you think?

Alex

[mfonville]

It is not a deal breaking or blocking issue per se, but since on Marshmallow all APKs need to be fully intact (and thus the libs are not allowed to be removed) it takes much more space on /system/ if the libs are stored twice, once within the apk, and once pre-extracted.

Also it negatively influences our "For Stock" flashing method, that does support flashing on stock ROMs. (Since that one would then expect different behaviour).

I am sure Google must have included it in the code somewhere, since it is a big change to the core of the basesystem. It is very hard to believe this change to be propietary.

We could also look how other vendors do this on their 6.0 images.

[deadman96385]

I think i have found the code that allows libraries to be in apk's as well as a way to enable it on aosp builds i am build a build now with my changes i will let you know on the results. Also on CM based roms there privacy guard auto allows google apps certain base permissions maybe look there for other ways of fixing it.

[TheCrazyLex]

@deadman96385 Awesome man! :)

Please share the code so I can take a look :)

I think the permission issue is closed now, the code is stable and working.

[deadman96385]

Here is the commit for it, i am currently making a test build with the default set to false https://github.com/CyanogenMod/android_frameworks_base/commit/ff193d642eea7128faad837d19e347cd25212c27

[cryptomilk]

Why not check the signature of the apk in additon? I increases the security and isn't hard to do ...

[mfonville]

I got an extra update for what @deadman96385 did find. That method requires that the libs are not deflated, but 'stored' in the APK. Something that is done with the APKs on the nexus image, but not with APKS from the Play Store. So I will add something to the build script to fix that.

[TheCrazyLex]

@mfonville @deadman96385 Good work guys :)

@gladiac I will look into it.

[mfonville]

This will make sure that the libs are stored uncompressed within the APK, when building for Marshmallow: https://github.com/opengapps/opengapps/commit/90b34ea1d91aa35ebe1853fb6f74cfdb4489889f

[TheCrazyLex]

@mfonville does that mean your commit in combination with the cm commit resolves the issue?

[deadman96385]

The commit is a aosp commit and his new commit fixes it so that no change is need to the rom for it work i have just tested it.

[TheCrazyLex]

@deadman96385 So that means we are good to go and this issue can be marked as resolved :)

Alex

[deadman96385]

Yep you are corrected

[TheCrazyLex]

Ok, last thing to do which isn't critical would be add more security by verfifying the signature.

Maybe @gladiac can help, as he said it is easy to do so :)

[mfonville]

Thank you all for your hard work :-) and good collaboration between all parties! The bug can finally be closed

[mfonville]

@TheCrazyLex

Hey Alex,

could you maybe make for us a beta patch that also adds the following permissions for the Google Dialer (com.android.dialer)? Because we would like to try if the Google Dialer would work on Stock ROMs, but we think that the lack of permissions is obstructing it to work, and we would like to try it:

   <uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
   <uses-permission android:name="android.permission.CALL_PHONE"/>
   <uses-permission android:name="android.permission.READ_CONTACTS"/>
   <uses-permission android:name="android.permission.WRITE_CONTACTS"/>
   <uses-permission android:name="android.permission.READ_CALL_LOG"/>
   <uses-permission android:name="android.permission.WRITE_CALL_LOG"/>
   <uses-permission android:name="android.permission.READ_PROFILE"/>
   <uses-permission android:name="android.permission.MANAGE_ACCOUNTS"/>
   <uses-permission android:name="android.permission.GET_ACCOUNTS"/>
   <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>
   <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"/>
   <uses-permission android:name="android.permission.INTERNET"/>
   <uses-permission android:name="android.permission.NFC"/>
   <uses-permission android:name="android.permission.PROCESS_OUTGOING_CALLS"/>
   <uses-permission android:name="android.permission.READ_PHONE_STATE"/>
   <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
   <uses-permission android:name="android.permission.MODIFY_PHONE_STATE"/>
   <uses-permission android:name="android.permission.WAKE_LOCK"/>
   <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
   <uses-permission android:name="android.permission.WRITE_SETTINGS"/>
   <uses-permission android:name="android.permission.USE_CREDENTIALS"/>
   <uses-permission android:name="android.permission.VIBRATE"/>
   <uses-permission android:name="android.permission.READ_SYNC_SETTINGS"/>
   <uses-permission android:name="com.android.voicemail.permission.ADD_VOICEMAIL"/>
   <uses-permission android:name="com.android.voicemail.permission.WRITE_VOICEMAIL"/>
   <uses-permission android:name="com.android.voicemail.permission.READ_VOICEMAIL"/>
   <uses-permission android:name="android.permission.ALLOW_ANY_CODEC_FOR_PLAYBACK"/>
   <uses-permission android:name="com.android.launcher.permission.INSTALL_SHORTCUT"/>
   <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>
   <uses-permission android:name="com.google.android.providers.gsf.permission.READ_GSERVICES"/>
   <uses-permission android:name="android.permission.BROADCAST_STICKY"/>
   <uses-permission android:name="android.permission.INTERACT_ACROSS_USERS"/>
   <uses-permission android:name="android.permission.STOP_APP_SWITCHES"/>
   <uses-permission android:name="android.permission.CONTROL_INCALL_EXPERIENCE"/>
   <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>

Thanks!