Consider support for Android VMs

[andrewdavidwong]

Over the years, many Qubes users on the mailing lists have expressed the desire to be able to run some kind of Android VM (e.g., HVM) under Qubes OS.

[adrelanos]

May I suggest to move discussions on read-only elsewhere. I don't see how it is specifically related to Android VMs.

[adrelanos]

@adrelanos The solution was to enable software rendering because Qubes doesn't support OpenGL within VMs. I'm surprised that you managed to run it, the Whonix guide doesn't instruct to enable software rendering. snap set anbox software-rendering.enable=true snap restart anbox.container-manager

I didn't need to enable software rendering.

The command from Instructions for enabling Anbox software rendering probably won't work for Whonix Anbox guide as it does not use snap and it shouldn't use snap either because that would break recommendation always verify signatures since snap does not verify gpg signatures (yet?).

[v6ak]

I have looked at Anbox under Debian. I have tried the snap version (which is OK for just trying in a DVM and is likely more fresh than the package in Debian repo). It has security updates from Jan 2017, i.e., over 3 years old. That's no go for me. Maybe there can be few reasonable exceptions, but other than that, it is simply unacceptable.

[thestinger]

Worth noting that Anbox has the Android sandbox and security model mostly disabled, and security within a Qubes guest is still important. It's contained from impacting the rest of the system but I don't think you should abandon having a proper security model in guests. It makes far more sense to run an x86 build of AOSP in the guest with the security model intact and full app compatibility rather than a partial implementation.

[thestinger]

I'd also recommend using AOSP, which has x86 support, and just needs a properly configured kernel, etc. rather than a fork based on an out-of-date version that's rolling back the security model. I'm sure Android x86 has some useful patches, but unfortunately, it's largely focused on things counter to the goals and needs for this. An installer isn't needed and as people mentioned earlier the images could be updated elsewhere rather than trying to be compatible with the proper A/B update system + verified boot, or hacking together something much worse. Just keep things simple, use the latest code, and work on getting input devices and then further integration working smoothly. It's going to involve forking a few AOSP repositories to do the integration and it's really best if it's based on the latest stable release and sticks as close as possible to that to accomplish the goal of compatibility and QubesOS integration rather than starting from a fork not aligned with the goals.

This is still on the radar of the GrapheneOS project, and we've done some initial research and work on it, but we need to find developers that can be hired to work on it. If you're interested in the development and maintenance of a production-oriented port of AOSP to QubesOS including putting together integration for clipboard support, etc. and you have a record of open source contributions and relevant experience (not necessarily with AOSP itself but Java / Kotlin app development experience would be good), contact us at contact@grapheneos.org. There is funding available to get someone a nice workstation and a small stipend to work on it, but we can't currently fully fund the develop work ourselves. There are likely organizations interested in this among those that we're in contact with though and I would expect that I could get someone full-time funding to work on this if they got things rolling. This is a target that we want to support for GrapheneOS, but the support for it would be maintained as a separate project without the other changes to AOSP, which would then be used by GrapheneOS to build for the QubesOS guest target.

[Linnxam]

@adrelanos Are you using an iGPU or external? Some people are encountering the same issues, last resort was to install it using snap. @v6ak @thestinger I'm aware of the security issues, that's why I cut networking to the VM. For those requiring connectivity this obviously won't work. Unfortunately it seems for now there's no other working Android VM solution where file and clipboard sharing work.

Now I'd like to verify whether or not the usage as outlined above would leave traces of VM content on disk, in my case e.g. a "Google Translate" app history. The 2 measures I took so far were setting the root volume to read-only and having the VM use its maximal capacity. I can't even create an empty text file within the VM anymore. Could I hash & compare whole VM volumes before and after execution, or how could this verification issue be approached? Not sure if this is on topic @marmarek. But I think some people opting for this temporary solution might be asking themselves the same questions.

[tabbyrobin]

This is still on the radar of the GrapheneOS project, and we've done some initial research and work on it, but we need to find developers that can be hired to work on it. If you're interested in the development and maintenance of a production-oriented port of AOSP to QubesOS including putting together integration for clipboard support, etc. [...] This is a target that we want to support for GrapheneOS, but the support for it would be maintained as a separate project without the other changes to AOSP, which would then be used by GrapheneOS to build for the QubesOS guest target.

I really hope this happens. I think GrapheneOS on Qubes could be very useful for low budget/low memory Qubes users (4-8gb RAM). Because android has a decent internal security model/compartmentalization, this way users could use fewer Qubes domains.

[mfc]

This is still on the radar of the GrapheneOS project, and we've done some initial research and work on it, but we need to find developers that can be hired to work on it. If you're interested in the development and maintenance of a production-oriented port of AOSP to QubesOS including putting together integration for clipboard support, etc. and you have a record of open source contributions and relevant experience (not necessarily with AOSP itself but Java / Kotlin app development experience would be good), contact us at contact@grapheneos.org. There is funding available to get someone a nice workstation and a small stipend to work on it, but we can't currently fully fund the develop work ourselves. There are likely organizations interested in this among those that we're in contact with though and I would expect that I could get someone full-time funding to work on this if they got things rolling. This is a target that we want to support for GrapheneOS, but the support for it would be maintained as a separate project without the other changes to AOSP, which would then be used by GrapheneOS to build for the QubesOS guest target.

hey @thestinger does this offer still stand? if so i will share with some potential interested folks. thanks!

[thestinger]

It's still open but we're quite busy with a lot of very high priority work and are stretched pretty thin right now. There are some much higher priority / time sensitive projects such as implementing memory tagging for hardened_malloc which we need to find people to do right now.

[tasket]

I just installed the 9.0r2 version of Android-x86 from the iso. It seems to work OK for 30-60min but overall is not very stable with 3GB RAM; maybe I'll try more.

[maltfield]

Any update on this? As more-and-more services require apps, it seems that security through compartimentalization can only be attained by having one phone per app. This, combined with the fact that secure Android ROMs like GrapheneOS only support recent models, means we're forced to buy & dispose-of several phones every few years.

Rather than buy & provision a new phone for every service, it seems obvious to just run a bunch of Android VMs in Qubes. But how?

[thestinger]

it seems that security through compartimentalization can only be attained by having one phone per app

You're mixing up one way to do something being the only way to do it. x86 virtualization is definitely not the best way to do it.

This, combined with the fact that secure Android ROMs like GrapheneOS only support recent models, means we're forced to buy & dispose-of several phones every few years.

GrapheneOS doesn't only support recent devices. It only officially supports devices which meet the security requirements including having full security patches for firmware, drivers, etc. The list of requirements is at https://grapheneos.org/faq#future-devices. We provide extended support for devices with less than 5 years of official support, but do not consider it official since devices with full security patches are not secure, even if that's the norm for laptops and desktops which no one seems to care about. The current generation devices receive support for 7 years, which is quite a lot longer than nearly anything else. How long do non-Chromebook, non-Macbook laptops typically get complete firmware updates? 2 years? Do those updates cover everything? They certainly don't provide comparable security features in the first place and an unserious approach to a lot of the security model, which applies to a lot of the x86 hardware/firmware ecosystem. Lots of features are implemented in name only with a blatantly insecure, incomplete approach like the x86 "secure boot".

[thestinger]

Also, another reminder that Android-x86 is an unofficial, insecure project which appears to be dead. It's not the official Android x86 support. The approach it takes is also thoroughly incompatible with making a robust and secure OS.

[thestinger]

The past 3 generations of devices we support have hardware accelerated virtualization support which is currently used for ephemeral micro VMs for sandboxing within the OS including for compiling bytecode to native code. We plan to add support for running Android apps within minimal GrapheneOS VMs.

The latest device generation has support for DisplayPort alternate mode which currently supports an initial primitive desktop mode that's rapidly maturing. We plan to make that more useful by supporting running desktop applications via virtualization. We expect desktop mode to be very mature by the end of 2024 so we'd like to get this implemented this year.

We currently haven't thought of a use for us to use the protected VM capabilities of pKVM by taking advantage of attestation for this but perhaps there's an interesting use case for it. We already have a lot of experience with the main hardware attestation API due to our Auditor app, and we got them to add explicit support for pinning via the app generated attest key feature. We don't yet have experience using the pKVM attestation integration but it's likely based on that and quite straightforward.

[DemiMarie]

it seems that security through compartimentalization can only be attained by having one phone per app

You're mixing up one way to do something being the only way to do it. x86 virtualization is definitely not the best way to do it.

This, combined with the fact that secure Android ROMs like GrapheneOS only support recent models, means we're forced to buy & dispose-of several phones every few years.

GrapheneOS doesn't only support recent devices. It only officially supports devices which meet the security requirements including having full security patches for firmware, drivers, etc. The list of requirements is at https://grapheneos.org/faq#future-devices. We provide extended support for devices with less than 5 years of official support, but do not consider it official since devices with full security patches are not secure, even if that's the norm for laptops and desktops which no one seems to care about. The current generation devices receive support for 7 years, which is quite a lot longer than nearly anything else. How long do non-Chromebook, non-Macbook laptops typically get complete firmware updates? 2 years? Do those updates cover everything?

How many of these vulnerabilities get one anything beyond code execution in sys-net or sys-usb (explicitly untrusted in the Qubes OS threat model) or persistance in an already-compromised dom0 (but a compromised dom0 is considered Game Over)?

They certainly don't provide comparable security features in the first place and an unserious approach to a lot of the security model, which applies to a lot of the x86 hardware/firmware ecosystem. Lots of features are implemented in name only with a blatantly insecure, incomplete approach like the x86 "secure boot".

Proper secure boot is something that I at least would very much like to have in Qubes OS. There are at least three major difficulties, but this list is not exhaustive:

1. Hardware and firmware support, as you pointed out.
2. Storing configuration data (such as the location of the OS image) that must be available before the root filesystem is decrypted.
3. Preserving configurability and customizability. Qubes OS supports multiple desktop environments, including XFCE, KDE, i3, and AwesomeWM.

[adrelanos]

Secure Boot related:

• https://github.com/QubesOS/qubes-issues/issues/8206
• https://github.com/QubesOS/qubes-issues/issues/4371

[mfc]

temporarily locking conversation as it is gone totally off the rails. will let someone smarter like @andrewdavidwong deal with whatever else is needed.

[bi0shacker001]

Not sure if this is still an issue, but there are instructions on how to set up templates which use sway with waydroid for an android vm. This'll probably get easier as qubes moves to wayland, maybe a waydroid session template (https://docs.waydro.id/faq/setting-up-waydroid-only-sessions) after the migration?

[maltfield]

I did, in fact, get waydroid working inside an Ubuntu HVM. This is from my personal notes from a few years ago

sudo apt-get install weston curl wget ca-certificates
wget https://repo.waydro.id/waydroid.gpg

# verify key fingerprint
gpg waydroid.gpg

mv waydroid.gpg /usr/share/keyrings/
echo "deb [signed-by=/usr/share/keyrings/waydroid.gpg] https://repo.waydro.id/ bookworm main" | tee /etc/apt/sources.list.d/waydroid.list
apt-get update
apt-get install waydroid

# open a wayland UI
weston

# in the wayland Window, open a terminal and in *that* terminal, type
sudo waydroid init
sudo systemctl restart waydroid-container
waydroid session start # make sure this is *not* root

# then in another terminal
waydroid app install F-Droid.apk

# prevent the screen from rotating https://github.com/waydroid/waydroid/issues/70
waydroid prop set persist.waydroid.multi_windows true

waydroid show-full-ui

But it's buggy as hell and, most importantly, waydroid contributors said that waydroid shouldn't be used for anything sensitive since it's horribly insecure.

I've been monitoring this thread specifically because I was hoping to find a way to run an Android VM in Qubes that's actually secure (eg for extremely compartimentalized banking apps locked inside Qubes, not on my everyday personal phone)

The devs ITT above pointed me to BlissOS, but I didn't look into using it in Qubes yet

• https://blissos.org/

[bi0shacker001]

I did, in fact, get waydroid working inside an Ubuntu HVM. This is from my personal notes from a few years ago

# open a wayland UI
weston

# in the wayland Window, open a terminal and in *that* terminal, type
sudo waydroid init
sudo systemctl restart waydroid-container
waydroid session start # make sure this is *not* root

# then in another terminal
waydroid app install F-Droid.apk

# prevent the screen from rotating https://github.com/waydroid/waydroid/issues/70
waydroid prop set persist.waydroid.multi_windows true

waydroid show-full-ui

But it's buggy as hell and, most importantly, waydroid contributors said that waydroid shouldn't be used for anything sensitive since it's horribly insecure.

I've been monitoring this thread specifically because I was hoping to find a way to run an Android VM in Qubes that's actually secure (eg for extremely compartimentalized banking apps locked inside Qubes, not on my everyday personal phone)

(as a note, that prop doesn't stop the screen from rotating, it enables "seamless" mode. Apps will still try to rotate if you have a device with an accelerometer, it'll just break them when they do. You can run "wm set-fix-to-user-rotation disabled" to forcibly disable autorotate)

The security issues with waydroid are the same ones we deal with in other linux apps. They're ostensibly worse on qubes, as we give the user account passwordless root. That said, I'd say it's no more insecure than running any other code in a qube, as all applications could grab root immediately and effortlessly with sudo, if they wanted to. It'd still be contained by the qube itself.

[maltfield]

The security issues with waydroid are the same ones we deal with in other linux apps

My biggest concern in these Android Emualtors is the supply chain and the risk of obtaining inauthentic code that's been maliciously modified.

I'm not worried about my banking app taking over the host AndroidVM that was created just for the purpose of running this one "trusted" app.

I am worried about downloading a malicious copy of an Android Emulator that has spyware that can see what I'm doing in my banking app or, worse, steal my credentials or preform actions to my banking accounts on my behalf.

Most Android ROMs don't even sign their releases (LOS refused to sign their releases with PGP), and their build chains often include fetching unsigned code (anything with pip is a red flag, for example) so they're a target for watering hole attacks.

[bi0shacker001]

The security issues with waydroid are the same ones we deal with in other linux apps

My biggest concern in these Android Emualtors is the supply chain and the risk of obtaining inauthentic code that's been maliciously modified.

I'm not worried about my banking app taking over the host AndroidVM that was created just for the purpose of running this one "trusted" app.

I am worried about downloading a malicious copy of an Android Emulator that has spyware that can see what I'm doing in my banking app or, worse, steal my credentials or preform actions to my banking accounts on my behalf.

Most Android ROMs don't even sign their releases (LOS refused to sign their releases with PGP), and their build chains often include fetching unsigned code (anything with pip is a red flag, for example) so they're vulnerable to watering hole attacks.

Wouldn't we have that issue regardless of what solution we go with? Short of building android locally on our own device (which we CAN do, and waydroid will run the image), there aren't any solutions which require signed images, and nowhere really releases them. Unless you're referring to trusting the Qubes project to verify the android they ship in a template.

Android doesn't really run natively, it's already in a simulated environment. The only difference between Waydroid, Anbox, Android-x86, and my phone, is what interface layer everything's being virtualized in, and you'd need to pick one regardless.

[maltfield]

Unless you're referring to trusting the Qubes project to verify the android they ship in a template.

I think that would be the best-case for the UX. I would love for Qubes to ship with an "Android Template" along-side the "Fedora Template" and "Debian Template" and "Whonix Template"

As more-and-more identity providers go with Phone-only apps (I'm looking at banking apps and also the EU's eID legislation that is targeting making eID roll-out mandatory in EU Member States in 2026), I think this will be very important for many users -- being able to lock their banking and eID private keys inside their Qubes laptop instead of exposing it to their phones that are secured with shitty passwords (tapped-out in public dozens of times per day) and are easy to physically break & loose.

Short of building android locally on our own device (which we CAN do, and waydroid will run the image)

Do you have a guide to do this, but which focuses on security -- so we can actually verify the cryptographic signature of all the sourcecode (and all dependencies) that's downloaded before building?

[DemiMarie]

Unless you're referring to trusting the Qubes project to verify the android they ship in a template.

I think that would be the best-case for the UX. I would love for Qubes to ship with an "Android Template" along-side the "Fedora Template" and "Debian Template" and "Whonix Template"

As more-and-more identity providers go with Phone-only apps (I'm looking at banking apps and also the EU's eID legislation that is targeting making eID roll-out mandatory in EU Member States in 2026), I think this will be very important for many users -- being able to lock their banking and eID private keys inside their Qubes laptop instead of exposing it to their phones that are secured with shitty passwords (tapped-out in public dozens of times per day) and are easy to physically break & loose.

On at least Pixels, eID is in part handled by dedicated code running in the secure element chip.

[bi0shacker001]

Unless you're referring to trusting the Qubes project to verify the android they ship in a template.

I think that would be the best-case for the UX. I would love for Qubes to ship with an "Android Template" along-side the "Fedora Template" and "Debian Template" and "Whonix Template"

As more-and-more identity providers go with Phone-only apps (I'm looking at banking apps and also the EU's eID legislation that is targeting making eID roll-out mandatory in EU Member States in 2026), I think this will be very important for many users -- being able to lock their banking and eID private keys inside their Qubes laptop instead of exposing it to their phones that are secured with shitty passwords (tapped-out in public dozens of times per day) and are easy to physically break & loose.

If those are specifically your use cases, most secure android apps are reliant on google Safetynet, which we can neither pass nor spoof on pc. Heck, the days of phones spoofing that are long since over.

Short of building android locally on our own device (which we CAN do, and waydroid will run the image)

Do you have a guide to do this, but which focuses on security -- so we can actually verify the cryptographic signature of all the sourcecode (and all dependencies) that's downloaded before building?

Unfortunately, no. That's not something I've ever worried about when building android roms (i'm not overly concerned with a hostile party finding out how bad I am at fruit ninja, for instance, That's what I use waydroid for, gaming and entertainment)

[ypid]

@maltfield Instead of this:

sudo apt-get install weston curl wget ca-certificates
wget https://repo.waydro.id/waydroid.gpg

# verify key fingerprint
gpg waydroid.gpg

mv waydroid.gpg /usr/share/keyrings/
echo "deb [signed-by=/usr/share/keyrings/waydroid.gpg] https://repo.waydro.id/ bookworm main" | tee /etc/apt/sources.list.d/waydroid.list
apt-get update
apt-get install waydroid

You could use extrepo like that:

sudo apt-get install weston ca-certificates extrepo
extrepo enable waydroid
apt-get update
apt-get install waydroid

Ref: https://salsa.debian.org/extrepo-team/extrepo-data/-/blob/master/repos/debian/waydroid.yaml?ref_type=heads

[thestinger]

Waydroid is based on an insecure fork of Android 11 and has most of the security model disabled. It doesn't have app sandboxing intact and doesn't protect the kernel from apps as Android does. It doesn't have SELinux support which is a major part of the Android application sandbox and the overall security model/approach including kernel attack surface reduction. Android depends on SELinux for the basic privacy and security properties to be intact. The main issue is lack of ongoing maintenance and security patches. Only Android 14 gets full security support, but they could at least be shipping the monthly security backports of High/Critical severity vulnerabilities for Android 11 and using AOSP instead of LineageOS.

[apparatius]

Waydroid is based on an insecure fork of Android 11 and has most of the security model disabled. It doesn't have app sandboxing intact and doesn't protect the kernel from apps as Android does. It doesn't have SELinux support which is a major part of the Android application sandbox and the overall security model/approach including kernel attack surface reduction. Android depends on SELinux for the basic privacy and security properties to be intact.

I don't think this will matter much for the default use case as AppVM in Qubes OS with only a single app per AppVM and not using multiple apps in the same AppVM.

The main issue is lack of ongoing maintenance and security patches. Only Android 14 gets full security support, but they could at least be shipping the monthly security backports of High/Critical severity vulnerabilities for Android 11 and using AOSP instead of LineageOS.

This one is an issue for sure.

[thestinger]

It means that a compromise in normally heavily sandboxed OS components is often going to be a compromise of the whole OS including the app. You may not care about the OS being secure from the app if the app sandbox is a VM but you should still care about the OS security. Being on Android 11 is bad enough due to years of missing Low/Moderate severity patches and security improvements but not having the monthly backports of High/Critical severity patches delivered in practice is much worse.

[apparatius]

It means that a compromise in normally heavily sandboxed OS components is often going to be a compromise of the whole OS including the app.

Maybe you're right. I'm not knowledgeable in how android works so I can only guess some things. Correct me if I'm wrong in guessing this situation:

I have only WhatsApp installed in android. I receive the image in WhatsApp and open it in WhatsApp. Android will open the image using some internal image viewing android component (or a WhatsApp component that is separately sandboxed from the other WhatsApp components). The image is malicious and can compromise the image viewing component but not the WhatsApp itself or other components. So if the image viewing android component is properly sandboxed then it'll only have access to this image viewing component data and won't be able to easily access valuable WhatsApp data.

[thestinger]

If it's opened in another app or handled with OS media handling, sure, that's how it works. WhatsApp likely handles a lot of the media handling with in-process libraries and it's up to them to do it in an isolatedProcess which they probably don't currently do even though it's very easy. None of the OS sandboxing including isolatedProcess is going to work properly without SELinux. SELinux is not an additional layer but rather a huge part of the basic privacy/security model in Android. It's used much differently from how desktop OSes use it.

[DemiMarie]

even though it's very easy

Is it? Using the IsolatedProcess API might well be easy, but replacing code that runs in-process with an IPC call is not generally easy. Is there something specific about Android that makes it possible to wrap a general media decoding API this way? If so, would it be better for this to be done by Android automatically, rather than each app writer having to do this manually?

[thestinger]

It's easy because it's all abstracted and uses the same API for communicating as a non-isolatedProcess service which are heavily used by apps already. Developers already have to deal with moving things to run in the background and this doesn't change much about it. The main portion of the app runs in a sandbox too and it's not that much different.

[bi0shacker001]

Waydroid is based on an insecure fork of Android 11 and has most of the security model disabled. It doesn't have app sandboxing intact and doesn't protect the kernel from apps as Android does. It doesn't have SELinux support which is a major part of the Android application sandbox and the overall security model/approach including kernel attack surface reduction. Android depends on SELinux for the basic privacy and security properties to be intact. The main issue is lack of ongoing maintenance and security patches. Only Android 14 gets full security support, but they could at least be shipping the monthly security backports of High/Critical severity vulnerabilities for Android 11 and using AOSP instead of LineageOS.

Just as a note: waydroid uses lineageos-based images, which DO receive security updates, and the waydroid updater does receive weekly update images

[thestinger]

Just as a note: waydroid uses lineageos-based images, which DO receive security updates, and the waydroid updater does receive weekly update images

Android 11 support ended after February. Prior to that, Android 11 only received backports of the High/Critical severity vulnerabilities and only the subset deemed to be part of the standard Android security patches. It doesn't include the Low/Moderate severity patches including most privacy fixes, etc. LineageOS only provides a subset of that subset of standard security patches particularly since they make changes incompatible with security features which would be considered security vulnerabilities.

Android doesn't have an actively developed LTS branch for 12, 13 and 14 for use as a mobile OS but rather monthly security backports vendors are meant to apply to their forks. They stopped backporting most Moderate and Low severity patches and stopped assigning them a CVE. If you want all the patches, you need the April monthly release of Android 14 QPR2. For a while, they were listing all of the Low and Moderate severity AOSP patches as part of the Pixel bulletins but since they apply to all devices rather than only Pixels that was weird and they stopped listing them there so they aren't publicly listed beyond the commits now.

Waydroid loses not just a set of assorted security features like LineageOS but rather most of the userspace security model through having SELinux disabled. That includes not just the app sandbox but sandboxing throughout the OS and kernel attack surface reduction. It's not just most of the app sandbox but how media handling, HALs and everything else gets sandboxed within the OS. In the Android 4 and Android 5 era, it was an additional layer of security but it gradually became the main way low-level privacy and security controls are implemented including doing userspace-based enforcement of access to services based on SELinux. For example, without SELinux, any app can connect to the update_engine service since that kind of low-level IPC is controlled via userspace. LineageOS disables downgrade protection, so that's quite noteworthy.

In terms of kernel attack surface reduction, SELinux is how Android restricts access to eBPF to only bpfloader which only netd can use, how it restricts access to io_uring to fastbootd/snapuserd, how it does ioctl command filtering, etc. Additionally, a lot of the kernel options that are enabled by regular distributions including user namespaces, etc. are not allowed by Android and open up major new attack surfaces.

Also, the whole issue of building and distributing updates insecurely without proper signing and downgrade protection.

[DemiMarie]

How useful will Android VMs be in practice? So far, I have seen two uses:

1. As a replacement for the Android emulator, which doesn’t work on Qubes OS due to Xen not having nested virtualization support.
2. To run third-party (presumably proprietary) applications.

It might be possible to replace the Android emulator for development, though without the integrations that the Android emulator has. However, I am seriously unsure if it will be possible to use Android VMs to run third-party apps, mostly because Qubes OS only supports x86 whereas third-party apps will be compiled for Arm. Yes, Android apps are mostly written in JVM languages, but the NDK exists and real apps do use it. Also, any app that relies on hardware security features (like attestation, StrongBox, or Protected Confirmation) will find them unavailble.

[bi0shacker001]

How useful will Android VMs be in practice? So far, I have seen two uses:

1. As a replacement for the Android emulator, which doesn’t work on Qubes OS due to Xen not having nested virtualization support.

2. To run third-party (presumably proprietary) applications.

It might be possible to replace the Android emulator for development, though without the integrations that the Android emulator has. However, I am seriously unsure if it will be possible to use Android VMs to run third-party apps, mostly because Qubes OS only supports x86 whereas third-party apps will be compiled for Arm. Yes, Android apps are mostly written in JVM languages, but the NDK exists and real apps do use it. Also, any app that relies on hardware security features (like attestation, StrongBox, or Protected Confirmation) will find them unavailble.

Houdini and libndk are both extremely performant libraries for arm to x86 translation, so this isn't much of an issue in practice

[tonsimple]

How useful will Android VMs be in practice? So far, I have seen two uses:

1. As a replacement for the Android emulator, which doesn’t work on Qubes OS due to Xen not having nested virtualization support.
2. To run third-party (presumably proprietary) applications.

It might be possible to replace the Android emulator for development, though without the integrations that the Android emulator has. However, I am seriously unsure if it will be possible to use Android VMs to run third-party apps, mostly because Qubes OS only supports x86 whereas third-party apps will be compiled for Arm. Yes, Android apps are mostly written in JVM languages, but the NDK exists and real apps do use it. Also, any app that relies on hardware security features (like attestation, StrongBox, or Protected Confirmation) will find them unavailble.

I kind of think it would be actually pretty useful given a lot of "fancier" messengers out there are mobile-first nowadays, and quite a bit of general purpose computer stuff has better UX on mobile

I think loss of hardware-backed security features is mostly acceptable here, worst case scenario a "QubesDroid" would be as safe as other qubes VM which is reasonably secure (users would just have to update their posture accordingly)

It would certainly be a niche thing (a niche thing I like and desire hard enough to be subscribed to this discussion on-and-off from seemingly its very beginning) but I do think it will find "useful usecases" among Qubes users.

[thestinger]

It's possible to make a software-emulated implementation of the hardware security features where within the context of the VM they're enforced despite not having actual hardware backing outside of it.

[DemiMarie]

That is very much true. To use a trivial example, “The OS can only boot from this disk image, which it doesn’t have write access to,” is a valid verified boot implementation! It isn’t interesting on real hardware because it doesn’t allow updates, but in Qubes OS updates are handled out of band. Similarly, a separate VM running cryptographic software counts as a TEE where Android is concerned.

@thestinger: How bad would it be to allow all third-party applications (untrusted_app and isolated_app IIUC) the ability to make memory executable? The reason I ask is that apps users want to run will likely have native code compiled for Arm, and this code must be emulated. The emulation will likely make use of a JIT for performance reasons.

Houdini and libndk are both extremely performant libraries for arm to x86 translation, so this isn't much of an issue in practice

The only information I can find about houdini is that it is a closed-source interpreter, which I would not consider performant. qemu-user would likely be much faster if NDK support was added.

[thestinger]

That is very much true. To use a trivial example, “The OS can only boot from this disk image, which it doesn’t have write access to,” is a valid verified boot implementation! It isn’t interesting on real hardware because it doesn’t allow updates, but in Qubes OS updates are handled out of band. Similarly, a separate VM running cryptographic software counts as a TEE where Android is concerned.

In a sense, yeah, although it would need a different attestation root and it's not clear if it should really pretend that it's hardware backed but that might be required for app compatibility.

@thestinger: How bad would it be to allow all third-party applications (untrusted_app and isolated_app IIUC) the ability to make memory executable? The reason I ask is that apps users want to run will likely have native code compiled for Arm, and this code must be emulated. The emulation will likely make use of a JIT for performance reasons.

They can already normally do that but the JIT would presumably be external to the VM if not using an x86_64 Android OS. It would be too much work to make it possible to run arm64 apps on an x86_64 OS.

Houdini and libndk are both extremely performant libraries for arm to x86 translation, so this isn't much of an issue in practice

The only information I can find about houdini is that it is a closed-source interpreter, which I would not consider performant. qemu-user would likely be much faster if NDK support was added.

Wouldn't this just be done externally? Modifying the OS to have multiple architectures complicates things a lot. A lot of apps do run on x86 since the developers test in the x86 emulator but not all.

[DemiMarie]

That is very much true. To use a trivial example, “The OS can only boot from this disk image, which it doesn’t have write access to,” is a valid verified boot implementation! It isn’t interesting on real hardware because it doesn’t allow updates, but in Qubes OS updates are handled out of band. Similarly, a separate VM running cryptographic software counts as a TEE where Android is concerned.

In a sense, yeah, although it would need a different attestation root and it's not clear if it should really pretend that it's hardware backed but that might be required for app compatibility.

@thestinger: How bad would it be to allow all third-party applications (untrusted_app and isolated_app IIUC) the ability to make memory executable? The reason I ask is that apps users want to run will likely have native code compiled for Arm, and this code must be emulated. The emulation will likely make use of a JIT for performance reasons.

They can already normally do that but the JIT would presumably be external to the VM if not using an x86_64 Android OS.

QEMU full system emulation is far too slow and is also completely unsupported security-wise. I don’t consider it a feasible option except for testing or demonstration uses.

It would be too much work to make it possible to run arm64 apps on an x86_64 OS.

android-x86 and Waydroid already support it using the aforementioned interpreter.

Houdini and libndk are both extremely performant libraries for arm to x86 translation, so this isn't much of an issue in practice

The only information I can find about houdini is that it is a closed-source interpreter, which I would not consider performant. qemu-user would likely be much faster if NDK support was added.

Wouldn't this just be done externally? Modifying the OS to have multiple architectures complicates things a lot. A lot of apps do run on x86 since the developers test in the x86 emulator but not all.

Google’s emulator images come with this support (source: https://chgans.design.blog/2021/05/23/adding-arm-native-bridge-to-the-aosp11-x86-emulator/). The question is how to provide an open source implementation that can be used here.

[unman]

On Sat, Mar 30, 2024 at 10:08:00AM -0700, Daniel Micay wrote: [quote] Waydroid is based on an insecure fork of Android 11 and has most of the security model disabled. It doesn't have app sandboxing intact and doesn't protect the kernel from apps as Android does. It doesn't have SELinux support which is a major part of the Android application sandbox and the overall security model/approach including kernel attack surface reduction. Android depends on SELinux for the basic privacy and security properties to be intact. The main issue is lack of ongoing maintenance and security patches. Only Android 14 gets full security support, but they could at least be shipping the monthly security backports of High/Critical severity vulnerabilities for Android 11 and using AOSP instead of LineageOS. [/quote]

Daniel - is there any thing you would recommend in place of Waydroid for use in Qubes?

[DemiMarie]

For context: I think it is reasonable to assume that the apps being run are not trusted, and that each app is in a separate VM.

[emanruse]

A more recent discussion which referenced the current issue:

https://forum.qubes-os.org/t/android-template-qubes/28271/