Add USB device access portal

hadess commented 6 years ago

TBD

matthiasclasen commented 6 years ago

the way forward for these is for some application that needs these devices to step up and get a design for a portal, propose a portal api, and implement it. I'm happy to assist with these steps, but I'm unlikely to produce more portals all by myself without concrete involvement from api consumers.

hadess commented 6 years ago

the way forward for these is for some application that needs these devices to step up and get a design for a portal, propose a portal api, and implement it. I'm happy to assist with these steps, but I'm unlikely to produce more portals all by myself without concrete involvement from api consumers.

I'm getting the process started now.

hadess commented 6 years ago

Prior art

iOS "External Accessory" framework: https://developer.apple.com/documentation/externalaccessory https://developer.apple.com/library/archive/featuredarticles/ExternalAccessoryPT/Introduction/Introduction.html#//apple_ref/doc/uid/TP40009502
Android USB host overview: https://developer.android.com/guide/topics/connectivity/usb/host
USB device access for Chrome Apps: https://developers.chrome.com/apps/app_usb

On iOS, a device will declare which protocols it speaks (less "USB mass-storage" and more com.Hwmanufacturer.MassStorage), and apps will list which protocols they support. Whether apps are allowed to enumerate those devices is gated through the App Store, whether an app can access a specific device is done at the OS level ("App blah wants to access foo").

On Android, the process is similar. The application must declare which devices (USB vendor and product IDs), or which class/subclass (all headsets, all mass-storage devices) ahead of time, in its manifest. Enumeration is allowed within those boundaries, authorisation to access is then requested. It seems that Android doesn't handle only allowing certain endpoints to be accessed.

Chrome OS/Chrome Apps works similarly to Android.

Draft design

supported devices get listed in the Flatpak manifest, or in a separate file shipped in the Flatpak. We need to be able to access it from outside the sandbox to show it somewhere like the Privacy settings panel, or GNOME Software. This avoids applications being able to fingerprint users and devices.
add a portal that uses that device list, and filters out the rest of the devices for enumeration. This doesn't require authentication.
add a portal to open a particular device. If the enumeration is filtered on interface class, disallow claiming any interface but the one that was filtered on. This would trigger an authorisation dialogue. Ideally, the portal would be able to open devices without needing them to be accessible by the user itself. Do we want to make this a polkit authorisation instead of a Portal-level one? Do we also want to allow kernel drivers to be detached?
modify libusb to use that portal when run inside a flatpak
add a per-app toggle to allow/disallow USB access, which would just make libusb say that there's no supported devices.

alexlarsson commented 6 years ago

For listing the usb devices supported, maybe we can use the --add-policy=SUBSYSTEM.KEY=VALUE generic flatpak permissions. These are copied into the metadata file and will be accessible in /.flatpak-info for the running instance.

However, i'm not sure how this prohibits fingerprinting? Surely you can just list a shitload of devices? Would make the finger-printer more obvious though...

For the per-app permissions, we can store those in the permissions store.

Other than that, this sounds like a plan to me, but the devil is gonna be in the details.

hadess commented 6 years ago

For listing the usb devices supported, maybe we can use the --add-policy=SUBSYSTEM.KEY=VALUE generic flatpak permissions. These are copied into the metadata file and will be accessible in /.flatpak-info for the running instance.

Ideally, this would be imported into the manifest by flatpak-builder, so that we can have the applications programmatically list those identifiers during the build. A static list inside the manifest is probably fine if you support a single type of device, or just a few devices, but that list could get long (in the tens) pretty easily.

However, i'm not sure how this prohibits fingerprinting? Surely you can just list a shitload of devices? Would make the finger-printer more obvious though...

You'd need to list about 130k items if you wanted to list every possible VID:PID combinations. The user or the repository owner would surely notice.

For the per-app permissions, we can store those in the permissions store.

OK.

Other than that, this sounds like a plan to me, but the devil is gonna be in the details.

As always :)

hadess commented 6 years ago

You'd need to list about 130k items

Sorry, it's about 4 billion :)

bl33pbl0p commented 5 years ago

Slightly offtopic but with the potential of simplifying the implementation, recently device namespaces support made it into the kernel (so a privileged process with CAP_SYS_ADMIN can tag uevents and send to the correct network namespace owned by a user namespace) and then the device manager running inside that or just a little process listening for those could act accordingly. I don't know if this is useful for the portals implementation but I guess this might be interesting for, say, systemd, which could grow support to expose this to other users as some API that flatpak could benefit from. Ignore if this is irrelevant (perhaps this could be something flatpak developers can discuss with Lennart and work something out that is useful for everyone now that newer kernels support it).

hfiguiere commented 5 years ago

As an example, libgphoto2 has a set of udev rules and a script to properly setup permission on the devices with udev (on Fedora it is these two files):

/usr/lib/udev/check-ptp-camera
/usr/lib/udev/rules.d/40-libgphoto2.rules

Maybe we could provide a similar mechanism for the portal to determine which devices should be granted permission. With libgphoto2 the library is in the flatpak so this generated list could be used for the policy,

hadess commented 5 years ago

Maybe we could provide a similar mechanism for the portal to determine which devices should be granted permission. With libgphoto2 the library is in the flatpak so this generated list could be used for the policy,

Sorry, I don't understand to what that "mechanism" would be similar. udev permissions aren't usable or reliable for sandboxed applications, as applications can't rely on the host side having updated libraries and data files to give them access to devices, or the necessary updated libraries to detect devices at runtime (especially if that requires talking to the devices).

hadess commented 5 years ago

Slightly offtopic but with the potential of simplifying the implementation, recently device namespaces support made it into the kernel (so a privileged process with CAP_SYS_ADMIN can tag uevents and send to the correct network namespace owned by a user namespace) and then the device manager running inside that or just a little process listening for those could act accordingly.

udev events are not usable inside a sandbox as the kernel/user-space API and ABI is purposefully not designed to be forward or backwards compatible. Even if it were, filtering is done on the user-space side, so would leak information to the sandboxed application, and that would still leave the problem of device access (rather than device discovery).

rugk commented 4 years ago

Strongly related to supporting U2F/WebAuthn via USB, see: https://github.com/flatpak/xdg-desktop-portal/issues/989

CuriousTommy commented 4 years ago

Sorry, I don't understand to what that "mechanism" would be similar. udev permissions aren't usable or reliable for sandboxed applications

I don't think they asking for udev support per se, but for something that can give flatpak applications the ability to change the permissions of a USB device.

Due to the current limitations of flatpak, it is not possible to connect a real Wii Remote to the Dolphin emulator, unless you make a udev rule.

https://github.com/flathub/org.DolphinEmu.dolphin-emu/issues/44

It would be nice to have an API that can allow the user to change the permission of the device. So stuff, like the Wii Remote, can work out of the box.

5mdlc commented 4 years ago

Prior art
* iOS "External Accessory" framework:
  https://developer.apple.com/documentation/externalaccessory
  https://developer.apple.com/library/archive/featuredarticles/ExternalAccessoryPT/Introduction/Introduction.html#//apple_ref/doc/uid/TP40009502

* Android USB host overview:
  https://developer.android.com/guide/topics/connectivity/usb/host

* USB device access for Chrome Apps:
  https://developers.chrome.com/apps/app_usb
On iOS, a device will declare which protocols it speaks (less "USB mass-storage" and more com.Hwmanufacturer.MassStorage), and apps will list which protocols they support. Whether apps are allowed to enumerate those devices is gated through the App Store, whether an app can access a specific device is done at the OS level ("App blah wants to access foo").

On Android, the process is similar. The application must declare which devices (USB vendor and product IDs), or which class/subclass (all headsets, all mass-storage devices) ahead of time, in its manifest. Enumeration is allowed within those boundaries, authorisation to access is then requested. It seems that Android doesn't handle only allowing certain endpoints to be accessed.

Chrome OS/Chrome Apps works similarly to Android.

Draft design
* supported devices get listed in the Flatpak manifest, or in a separate file shipped in the Flatpak. We need to be able to access it from outside the sandbox to show it somewhere like the Privacy settings panel, or GNOME Software. This avoids applications being able to fingerprint users and devices.

* add a portal that uses that device list, and filters out the rest of the devices for enumeration. This doesn't require authentication.

* add a portal to open a particular device. If the enumeration is filtered on interface class, disallow claiming any interface but the one that was filtered on. This would trigger an authorisation dialogue. Ideally, the portal would be able to open devices without needing them to be accessible by the user itself. Do we want to make this a polkit authorisation instead of a Portal-level one? Do we also want to allow kernel drivers to be detached?

* modify libusb to use that portal when run inside a flatpak

* add a per-app toggle to allow/disallow USB access, which would just make libusb say that there's no supported devices.

Hi hadess, I am trying to move forward with issue 227 of xdg-desktop-portal, I'm analyzing how to modify libusb so that it verifies if it is running from flatpak or not. What method do you suggest to do it? I'm not sure that validating via dbus is the best option.

Thank you.

felipeborges commented 4 years ago

I'm analyzing how to modify libusb so that it verifies if it is running from flatpak or not. What method do you suggest to do it?

The existing APIs check for the existence of /.flatpak-info within the container.

g_file_test ("/.flatpak-info", G_FILE_TEST_EXISTS);

hadess commented 4 years ago

Hi hadess, I am trying to move forward with issue 227 of xdg-desktop-portal, I'm analyzing how to modify libusb so that it verifies if it is running from flatpak or not. What method do you suggest to do it? I'm not sure that validating via dbus is the best option.

Checking for /.flatpak-info is the best thing to do.

After that I think that the code in the client libusb will need to use D-Bus to get a file descriptor from the portal, and use something like libusb_wrap_sys_device() to open it as a device. Which means you can probably start working on an proof-of-concept by writing a test application that enumerates using the portal, and then opens the device through the portal, and see if libusb can still communicate with the device.

The main problem you're going to run into is device node ownership. Just chmod/chown the device node for now, until we figure out how that section would need to be implemented.

5mdlc commented 4 years ago

Hi hadess, I am trying to move forward with issue 227 of xdg-desktop-portal, I'm analyzing how to modify libusb so that it verifies if it is running from flatpak or not. What method do you suggest to do it? I'm not sure that validating via dbus is the best option.

Checking for /.flatpak-info is the best thing to do.

After that I think that the code in the client libusb will need to use D-Bus to get a file descriptor from the portal, and use something like libusb_wrap_sys_device() to open it as a device. Which means you can probably start working on an proof-of-concept by writing a test application that enumerates using the portal, and then opens the device through the portal, and see if libusb can still communicate with the device.

The main problem you're going to run into is device node ownership. Just chmod/chown the device node for now, until we figure out how that section would need to be implemented.

Thank you very much for your responses hadess and felipeborges.

Could I add as an extra condition that the process with id 1 is bwrap? to make sure it's really running from flatpak?

Thank you.

victortoso commented 4 years ago

Hi @5mdlc, I was wondering how is it going with this issue? Let me know if I can help you out, I'm quite interested on this too.

5mdlc commented 4 years ago

Hi @5mdlc, I was wondering how is it going with this issue? Let me know if I can help you out, I'm quite interested on this too.

Hi victortoso, at this moment, i have in pause this project, but you can to fork project and advance, any question you do have, i will pleasure to answer.

refi64 commented 3 years ago

Since this seems to still be unclaimed, I've started work on this, inspired by a mix of the current udev APIs and Android's permissions design.

One quick question: should this be part of the current device portal API or a different one (i.e. org.freedesktop.portal.Usb)? Right now I'm doing the former (extending the current API), but it would be pretty trivial to switch to the latter if preferred.

hadess commented 3 years ago

Since this seems to still be unclaimed, I've started work on this, inspired by a mix of the current udev APIs and Android's permissions design.

We need revoke support for USB first...

AsciiWolf commented 2 years ago

@refi64 Any update regarding your implementation? It is still not possible to use SDRs and other specific hardware with Flatpak apps because such devices almost always require custom udev rules to be installed (they work fine with Flatpak'ed SDR software after copying their udev rules manually into host /etc/udev). It would be great to have this working in Flatpak (at least as a portal to add/remove custom udev rules into the system).

elmarco commented 2 years ago

After that I think that the code in the client libusb will need to use D-Bus to get a file descriptor from the portal, and use something like libusb_wrap_sys_device() to open it as a device. Which means you can probably start working on an proof-of-concept by writing a test application that enumerates using the portal, and then opens the device through the portal, and see if libusb can still communicate with the device.

The main problem you're going to run into is device node ownership. Just chmod/chown the device node for now, until we figure out how that section would need to be implemented.

This might be of interest, I worked on a usbredir system helper recently: https://gitlab.freedesktop.org/elmarco/usbredir-rs/-/blob/main/usbredir-helper/src/main.rs

It opens the device, checking policykit permissions and hands off the device fd to the caller. Then libusb these days can wrap it. (some older libusb version couldn't do that, and spice-client-glib-usb-acl-helper did change device permissions, which could be problematic)

AsciiWolf commented 2 years ago

Another things that in many cases need custom udev rules are VR headsets.

hadess commented 2 years ago

It is still not possible to use SDRs and other specific hardware with Flatpak apps because such devices almost always require custom udev rules to be installed (they work fine with Flatpak'ed SDR software after copying their udev rules manually into host /etc/udev).

Another things that in many cases need custom udev rules are VR headsets.

Why can't those rules be upstreamed?

We've recently upstreamed rules to access media* nodes, protocol analysers and there's work on doing that for music production hardware. If things can't be upstreamed, there needs to be a reason, and SDR doesn't strike me as something security sensitive.

hadess commented 2 years ago

This might be of interest, I worked on a usbredir system helper recently: https://gitlab.freedesktop.org/elmarco/usbredir-rs/-/blob/main/usbredir-helper/src/main.rs

Something like this might work if it were something that libusb or systemd shipped directly. I would expect another layer or checking for allowed devices in a typical desktop use case (eg. not being able to have raw access to network or Bluetooth devices).

AsciiWolf commented 2 years ago

It is not always possible to ship these rules upstream. For example, the RTL based SDRs also work as DVB-T cards and some people use them this way. But after installing these SDR rules, they start working as SDR instead. On a regular GNU/Linux distro, these rules are installed as a part of a SDR software (and uninstalled when removing the software), but it can be done this way with Flatpak. And there are many other examples where it is not possible to ship the udev rules as a part of systemd/udevd.

edit: Looks like I was wrong and it actually is possible to ship these rules upstream (they do not contain any blacklists and do not break anything as mentioned in https://github.com/flatpak/xdg-desktop-portal/issues/227#issuecomment-961224458. Thanks!

hadess commented 2 years ago

For example, the RTL based SDRs also work as DVB-T cards and some people use them this way. But after installing these SDR rules, they start working as SDR instead. On a regular GNU/Linux distro, these rules are installed as a part of a SDR software (and uninstalled when removing the software), but it can be done this way with Flatpak.

By upstream, I meant definitely meant systemd. For DVB-T/SDR cards, just like for protocol analysers, there's nothing much to be broken by giving the user at the computer full access to the device, in a desktop setting. You would expect that to be blocked off on an appliance, not on a general-purpose computer.

I understand that this solution doesn't work for every type of device, but we need to start fixing problems upstream when they can be.

AsciiWolf commented 2 years ago

This is how the RTL-SDR rules do look like. I always thought that they also blacklisted the original kernel modules and did other things needed to make the RTL2832 devices work as an SDR. But it looks like that they don't. Maybe it is not needed anymore? Anyway, I have filled a ticket for upstreaming the udev rules here. I have also filled the same ticket for a similar SDR called Airspy here. Thanks for pointing this out! :-)

AsciiWolf commented 2 years ago

Regarding the VR headsets, various game controllers and other hardware devices that are shipped with Steam. @smcv Are there any plans to upstream (some of) these rules (into the systemd)?

smcv commented 2 years ago

Regarding the VR headsets, various game controllers and other hardware devices that are shipped with Steam. @smcv Are there any plans to upstream (some of) these rules (into the systemd)?

It's on my list, but it's a long list.

The main blocker is that I need to talk to Steam and Wine/Proton developers and write down a justification for why they need to access these devices with raw HID, as opposed to evdev.

brainchild0 commented 1 year ago

It appears that interest is growing for applications having low-level access to USB devices.

One area of increasing adoption is the various simplified and robust desktop distributions, for example Elemental OS and Endless OS, that are friendly to inexperienced users, but also offer an ability to operate more customizable operation systems in virtual machines behind the safe and friendly desktop.

Presently, the Gnome Boxes packaging in FlatPak supports the creation of virtual machines as such. Yet, lacking access to USB devices (see issue in Gnome Gitlab), the practical utility of these virtual machines in a desktop setting is severely limited.

For example, a Windows VM with a USB pass through would allow a user to repair errors on an NTFS removable store connected to the host, update firmware on a peripheral device such as headset, or help in any number of other real-world scenarios.

I have not followed the technical discussions regarding permissions, imposed either by the underlying system or by FlatPak. However, I feel it would be helpful for some improvement, hopefully in the foreseeable future, in overall support for broader use cases of applications installed under a Flatbak-based system.

Are any workarounds feasible, to consider adopting, that would allow site administrators or distribution maintainers to bypass certain security restrictions, in limited cases, in the interest of provisioning certain applications with elevated access to the system hardware, for serving some of the use cases that are presently stalled?

antistress commented 1 year ago

For example, a Windows VM with a USB pass through would allow a user to repair errors on an NTFS removable store connected to the host, or help in any number of other real-world scenarios.

repairing errors on an NTFS removable store is indeed a real-world scenario for me, and also updating maps on my GPS drive. Thanks !

brainchild0 commented 1 year ago

repairing errors on an NTFS removable store is indeed a real-world scenario for me, and also updating maps on my GPS drive.

Yes, it would be great if someone eventually would be able to create a Linux distribution featuring an installer that could convert an existing Windows system into a Linux system with the former remaining accessible as a VM. It might finally allow Linux to penetrate the residential market in a robust way as never before. (Of course the same was once said of dual boot.)

flatpak / xdg-desktop-portal