Exposing a tty serial device requires privileged and doesn't work

reconbot commented 8 years ago

Expected behavior

That I can expose a tty dev device from my mac to a linux container by running.

$ docker run --rm -it -v `pwd`:/app --device /dev/tty.usbmodem1421 node bash

Actual behavior

$ docker run --rm -it -v `pwd`:/app --device /dev/tty.usbmodem1421 node bash
docker: Error response from daemon: linux runtime spec devices: error gathering device information while adding custom device "/dev/tty.usbmodem1421": lstat /dev/tty.usbmodem1421: no such file or directory.

I can work past this with --privileged but I don't see it in /dev/

$ docker run --rm -it -v `pwd`:/app --device /dev/tty.usbmodem1421 --privileged node bash
root@5a6a7f6b8e58:/# ls /dev/tty.*
ls: cannot access /dev/tty.*: No such file or directory

According to the docs on docker run I shouldn't need --privileged.

Information

Full output of the diagnostics from "Diagnose & Feedback" in the menu

Docker for Mac: version: 1.12.3-beta29.3 (619507e)
OS X: version 10.12.1 (build: 16B2555)
logs: /tmp/E9C6B44E-7492-484A-A6EF-EE5BE163959F/20161105-153323.tar.gz
[OK]     vmnetd
[OK]     dns
[OK]     driver.amd64-linux
[OK]     virtualization VT-X
[OK]     app
[OK]     moby
[OK]     system
[OK]     moby-syslog
[OK]     db
[OK]     env
[OK]     virtualization kern.hv_support
[OK]     slirp
[OK]     osxfs
[OK]     moby-console
[OK]     logs
[OK]     docker-cli
[OK]     menubar
[OK]     disk

Page URL if this is a docs issue or the name of a man page https://docs.docker.com/engine/reference/run/#/runtime-privilege-and-linux-capabilities

Steps to reproduce the behavior

hook up a tty serial device such as an Arduino, usb serial dongle, modem, cash drawer, etc to your mac
Run the commands above to expose it's /dev/tty.usbXXXX device to the docker image
Look for the device

YunEr-Wang commented 3 years ago

Besides, if we use the VMWare, why do we still need the docker? :)

WolfgangFahl commented 3 years ago

Not a top priority ... see https://en.wikipedia.org/wiki/Kano_model https://en.wikipedia.org/wiki/Kano_model. grafik

This is a basic need and not negotiable. Dissatisfaction is a natural result of ignoring basic needs.

aleksandrkarabatov commented 3 years ago

Is tty exposing still the problem for MacOS Docker?

pharapeti commented 2 years ago

Yep.

HomeACcessoryKid commented 2 years ago

@justincormack, would you mind providing the view of Docker.com once more, like you did back in 2016? And potentially provide a subclass solution only, like e.g. for serial ports only? Thanks in Advance!

dentra commented 2 years ago

Any updates?

stephansturges commented 2 years ago

Oh wow this is disappointing... this is a major drawback for development on modern macs.

TheBengineer commented 2 years ago

I am also in need of a solution.

JelmerOffenberg commented 2 years ago

Bump, also looking for a solution for this.

lesterlo commented 2 years ago

Almost 6 years ago and still no solution. :(

lesterlo commented 2 years ago

In windows, there has a usbipd project to pass the USB devices from windows to WSL linux.

Is there an alternative or possibility to do this on mac and docker??

JetForMe commented 2 years ago

More than five years and still no solution.

lukepighetti commented 2 years ago

Cannot use zigbee2mqtt in docker on macOS due to the inability to passthrough a USB device. That means my momentum in IOT development has been blocked. Shame.

edagnin commented 2 years ago

It is SO infuriating to spend hours looking for a solution to a problem that is clearly very wide-spread only to eventually find a proposal that is said to have NO PRIORITY!

A long list of features under investigation on the docker roadmap and this isn't even included in that list.

Not a single developer @docker feels that this is an important thing to investigate? Come on.

lukepighetti commented 2 years ago

@stephen-turner is there any way we could get a fresh triage of this issue?

dannyyy commented 2 years ago

I'm also very eager to get this feature. Docker seems to focus on paying customers and to offer more Business services. In my opinion features only few people are locking for. But buzzword as SSO, Corporate Policy, ... still sells licenses, as the top management with no technical background finds it kinda cool. Our company decided to avoid docker at all as long as the business model is totally crappy and basic features don't even get on the shortlist.

There are alternatives. For the small and for the big. Bye bye Docker.

mlawson00 commented 2 years ago

This is a big problem for me, please sort it out Docker..

mrjoboto commented 2 years ago

+1, this is also preventing me from using Docker for my development workflow.

franciswertz commented 2 years ago

+1. Had planned to run/test my rtl-sdr container environment while traveling. Should have left the SDR dongle at home!

leppaott commented 2 years ago

--device=[] | Allows you to run devices inside the container without the --privileged flag.

But the --privileged is needed?

Same problem on Docker Desktop for Linux!

docker run --rm -it --device /dev/ttyACM0:/dev/ttyACM0:rwm --privileged image:latest ls /dev/ttyACM0
ls: cannot access '/dev/ttyACM0': No such file or directory

For my ad-hoc uses as we have simple Docker-compose setup, I got past this error using https://github.com/containers/podman-compose

Where the compose looks like:

 devices:
      - /dev/ttyACM0:/dev/ttyACM0

tomfat commented 2 years ago

Wa-oh, what a long thread.

Any plan to support USB PassThrough for hyperkit now? Or do we still have to use virtual box as backend for this feature, since the following repo is no longer maintained. https://github.com/boot2docker/boot2docker/

bsodmike commented 2 years ago

Any updates on this?

cosmikwolf commented 1 year ago

OMG DOCKER WHY!? is docker a dead project? hmmm...

thefill commented 1 year ago

OMG DOCKER WHY!? is docker a dead project? hmmm...

looks like it ^^

lesterlo commented 1 year ago

UTM also said the USB passthrough is not supported by apple's Virtualization.framework.

Maybe we need to wait unitil apple fix this.

https://github.com/utmapp/UTM/issues/3778#issuecomment-1067948682

JetForMe commented 1 year ago

How is it that Parallels can do this? It is able to attach to both USB-serial adapters as USB devices inside the hosted OS, or it can emulate a hardware serial port by connecting to a serial port on the host OS.

Also, it sure seems Virtualization.framework supports serial ports: https://developer.apple.com/documentation/virtualization/serial_ports

vonnieda commented 1 year ago

I've been trying to find any kind of possible workaround for this as it is the only thing standing in the way of me Dockerizing my entire embedded workflow. I've investigated USBIP, tty0tty, socat, gensio, ser2net, vtty, ttynvt, tty2com, and a dozen others.

None of these will do the trick as of today because they all require a kernel module to be loaded to create a true TTY (with line control) rather than a PTY.

For what it's worth, if you just need a data stream that sorta looks like a serial port and no line control you can probably just use socat with a PTY. I need RTS and DTR for reset signalling and this requires a TTY.

So, does anyone have any pointers on how to load a kernel module in Docker's LinuxKit VM? If I can get one of the above loaded, or FUSE, or a kernel module of my own, I can at least provide a workaround, but I've had no luck finding resources on that.

Tosko4 commented 1 year ago

Same issue here :(

lukepighetti commented 1 year ago

I really hope someone links up with @vonnieda since he seems uniquely positioned to contribute to this issue

iot49 commented 1 year ago

How is it that Parallels can do this? It is able to attach to both USB-serial adapters as USB devices inside the hosted OS, or it can emulate a hardware serial port by connecting to a serial port on the host OS.

@JetForMe How do this with Parallels? lsusb lists the device inside the container, but I cannot find it in /dev.

JetForMe commented 1 year ago

@JetForMe How do this with Parallels? lsusb lists the device inside the container, but I cannot find it in /dev.

In the Parallels VM window chrome there's a USB icon in the upper-right. Click that, and you get a list of USB devices attached to the host system. If you select one of those (and it's not in use by the host), it will attach to the VM being hosted. Should show up in lsusb then.

iot49 commented 1 year ago

@JetForMe In the Parallels VM window chrome there's a USB icon in the upper-right. ...

That part is working (checked with an Ubuntu VM, device shows up as /dev/ttyACM0).

But when I spin up ubuntu in a container (docker run --privileged ...), lsusb lists the device, but it does not show up in /dev. How can I "mount" it? I've tried creating an udev rule, but did not get that right

JetForMe commented 1 year ago

That's the subject of this bug. At least, Docker hosted on macOS can't seem to do this. Are you saying you're running Docker inside the VM? And the VM OS is Linux? Not sure if that's that case.

I merely mentioned that Parallels seems to have no issue sharing USB devices with the guest OS, so Docker has no excuses for not being able to do it.

iot49 commented 1 year ago

I've installed docker-machine following the instructions at https://github.com/Parallels/docker-machine-parallels/. This installs a special parallels VM (not docker desktop).

Since, as you point out, parallels can expose usb devices in its VMs, I figure they should also be available in the docker containers that use the parallels VM.

lsusb in the container seems to confirm this: it does list the device (with the correct VID:PID), suggesting it's available in the container. But I have not figured out how to expose it as a device in /dev (perhaps just a question of setting Linux up correctly, e.g. with a udev rule).

If this last problem can be solved, I can containers with usb passthrough on the Mac - albeit at the cost of a Parallels Pro subscription.

microboym commented 1 year ago

@iot49

If this last problem can be solved, I can containers with usb passthrough on the Mac - albeit at the cost of a Parallels Pro subscription.

Have you attempt to run docker-machine on virtual box (which is free to use)? https://dev.to/rubberduck/using-usb-with-docker-for-mac-3fdd

microboym commented 1 year ago

May you should add -v /dev:/dev or something like --device /dev...

@JetForMe In the Parallels VM window chrome there's a USB icon in the upper-right. ...

That part is working (checked with an Ubuntu VM, device shows up as /dev/ttyACM0).

But when I spin up ubuntu in a container (docker run --privileged ...), lsusb lists the device, but it does not show up in /dev. How can I "mount" it? I've tried creating an udev rule, but did not get that right

0xAl3xH commented 1 year ago

Wow, just found this issue after all these years and am surprised this has not gotten much attention from the devs despite the number of people it affects

h0jeZvgoxFepBQ2C commented 1 year ago

This should be really fixed :(

HomeACcessoryKid commented 1 year ago

Hey, exciting new insight, at least for me, which I bumped into via RancherDesktop They have just embraced the Apple Virtualization framework on macOS (VZ). This then supports Virtual I/O Device (VIRTIO) which could fix this serial port issue.

Not that it is there today, but RancherDesktop now offers VZ as experimental feature and it should be a small step to serial port support. Or am I missing something?

Next question is whether Docker will do the same, considering they support VZ already for some time.

@justincormack The high hanging fruit hangs a lot lower now, please come pick it?

saillingaway commented 1 year ago

Ended up here after a lot of troubleshooting and thinking I was missing something just cause I'm new to Docker, but this is a bummer. Hoping with what @HomeACcessoryKid pointed out we may see an update on this soon.

heanzyzabala commented 1 year ago

Same issue here when running zigbee2mqtt

Abcmsaj commented 11 months ago

Was completely pointless me buying a zigbee adapter to run on my Mac then...! Will be returning it until this is fixed (if ever...!)

quincarter commented 11 months ago

Any update on if this feature will get implemented in docker? USB Passthrough?

I also found this roadmap issue.

https://github.com/docker/roadmap/issues/511

Semmu commented 11 months ago

just a PSA, there are other alternatives for running containers on a Mac, for example:

https://github.com/abiosoft/colima
https://github.com/kubernetes/minikube (tutorial to use it for docker: https://itnext.io/goodbye-docker-desktop-hello-minikube-3649f2a1c469 )
and this one: https://macoscontainers.org/

i dont know whether any of these support USB passthrough as i dont need this feature anymore (solved my problem by using a different machine), but someone could experiment with these and maybe find an alternative solution to this (very old) issue

iot49 commented 11 months ago

I had no luck with Colima

On Sun, Dec 3, 2023 at 15:04 Laszlo Boros @.***> wrote:

just a PSA, there are other alternatives for running containers on a Mac, for example:

https://github.com/abiosoft/colima

https://github.com/kubernetes/minikube (tutorial to use it for docker: https://itnext.io/goodbye-docker-desktop-hello-minikube-3649f2a1c469 )

and this one: https://macoscontainers.org/

i dont know whether any of these support USB passthrough as i dont need this feature anymore (solved my problem by using a different machine), but someone could experiment with these and maybe find an alternative solution to this (very old) issue

— Reply to this email directly, view it on GitHub https://github.com/docker/for-mac/issues/900#issuecomment-1837630211, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATHYSZJU74S6FA7AITHYSHDYHUAPXAVCNFSM4CVJOKW2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBTG43DGMBSGEYQ . You are receiving this because you were mentioned.Message ID: @.***>

dracoventions commented 10 months ago

Another PSA: https://christopherjmcclellan.wordpress.com/2019/04/21/using-usb-with-docker-for-mac/ describes how to USB forward to a docker daemon run in VirtualBox on macOS. I haven't personally tried yet.

lmapii commented 10 months ago

Another PSA: https://christopherjmcclellan.wordpress.com/2019/04/21/using-usb-with-docker-for-mac/ describes how to USB forward to a docker daemon run in VirtualBox on macOS. I haven't personally tried yet.

This uses docker-machine which is now also deprecated. Had a running setup with that in 2019 as well, now docker-machine is gone too. See https://github.com/docker/roadmap/issues/245.

rubberduck203 commented 10 months ago

I’m actually the author of that blog. I’ve not tested that method recently and even at the time it only really worked with certain uart chips. Basically, use at your own caution.

rubberduck203 commented 10 months ago

I also don’t know what the odds of this are, but this xkcd dropped today.

https://xkcd.com/2881

reconbot commented 10 months ago

I thought I muted this thread long ago but you just made me laugh at loud, so I guess I'm glad I didn't. I hope everyone here is making cool stuff and having fun doing it.

-Francis

docker / for-mac