Closed nashant closed 3 years ago
Same issue: `pi@raspberrypi:~ $ sudo docker plugin ls ID NAME DESCRIPTION ENABLED c9ff714bd90b devplayer0/net-dhcp:latest Docker host bridge DHCP networking false
pi@raspberrypi:~ $ sudo docker plugin enable c9ff714bd90b Error response from daemon: dial unix /run/docker/plugins/c9ff714bd90b282648abed6a833812ad1a6e56d5cdfa1270a80046c4acc98eae/net-dhcp.sock: connect: no such file or directory`
@sohojmanush your problem there is that the plugin is not built for ARM, you have to build from source. If you enable debug logging on your docker daemon you'll see it exec format
error, meaning wrong architecture.
Having built from source my syslog with docker daemon debug logging on says this:
Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.984271891Z" level=debug msg="Calling HEAD /_ping"
Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.988903534Z" level=debug msg="Calling POST /v1.41/plugins/devplayer0/net-dhcp:latest/enable?timeout=120"
Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.995545011Z" level=debug msg="bundle dir created" bundle=/run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 module=libcontainerd namespace=plugins.moby root=/var/lib/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/rootfs
Mar 29 11:38:33 pi2 containerd[1691]: time="2021-03-29T11:38:33.168091953Z" level=info msg="starting signal loop" namespace=plugins.moby path=/run/containerd/io.containerd.runtime.v2.task/plugins.moby/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 pid=4425
Mar 29 11:38:33 pi2 systemd[1]: Started libcontainer container 0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.
Mar 29 11:38:33 pi2 systemd[1]: run-docker-runtime\x2drunc-plugins.moby-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5-runc.WqBeW5.mount: Succeeded.
Mar 29 11:38:33 pi2 systemd[3246]: run-docker-runtime\x2drunc-plugins.moby-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5-runc.WqBeW5.mount: Succeeded.
Mar 29 11:38:33 pi2 dockerd[1768]: time="2021-03-29T11:38:33.670772855Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/create
Mar 29 11:38:33 pi2 dockerd[1768]: time="2021-03-29T11:38:33.783955044Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/start
Mar 29 11:38:38 pi2 systemd-resolved[1572]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Mar 29 11:38:38 pi2 systemd-resolved[1572]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Mar 29 11:38:43 pi2 dockerd[1768]: time="2021-03-29T11:38:43Z" level=error msg=" * Running on unix:///run/docker/plugins/net-dhcp.sock (Press CTRL+C to quit)" plugin=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.289491267Z" level=debug msg="error net dialing plugin: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused"
Mar 29 11:38:46 pi2 systemd[1]: runc-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.scope: Succeeded.
Mar 29 11:38:46 pi2 systemd[1]: runc-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.scope: Consumed 5.418s CPU time.
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.515876811Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/exit
Mar 29 11:38:46 pi2 containerd[1691]: time="2021-03-29T11:38:46.651446243Z" level=info msg="shim disconnected" id=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5
Mar 29 11:38:46 pi2 containerd[1691]: time="2021-03-29T11:38:46.651963637Z" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.651788325Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/delete
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.651977178Z" level=info msg="ignoring event" container=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698413146Z" level=debug msg="Clean shutdown of plugin"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698807779Z" level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused" error_type="*net.OpError" module=api
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698996164Z" level=error msg="Handler for POST /v1.41/plugins/devplayer0/net-dhcp:latest/enable returned error: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.699351943Z" level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused" error_type="*net.OpError" module=api
I can see some DNS requests being made by the plugin for /.home
on my DNS server (.home
being my domain). That's never gonna work and I'm not sure why it would be doing that, but I don't think that's the issue because the same happens for amd64 machines and they work fine.
@nashant , thank you for clarifying I was suspecting the same.
Yep, this is an issue with the plugin being currently x86_64
only. Unfortunately I do not have the time to work on this plugin at the moment, my plan is to rewrite the whole thing in a month or so when I graduate from university :)
So I get why the install won't work, but the build from source should be fine because python:3-alpine has an arm64 image, no? Could you tell me how I can get the debug logs for the plugin itself? I'm happy to create a patch
Also, if I docker run --rm --name netdhcp -v /run/docker/plugins:/run/docker/plugins devplayer0/net-dhcp:rootfs
then it works. I can then
# curl -XPOST --unix-socket /run/docker/plugins/net-dhcp.sock http://localhost/NetworkDriver.GetCapabilities
{"ConnectivityScope":"global","Scope":"local"}
And if I manage to get to the socket in /run/docker/plugins/<hash>/net-dhcp.sock
quickly enough then it works too. Could there be a timeout thing where it just takes longer to get running on a pi?
Here is another one https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c
Here is another one https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c
That link doesn't work Edit: I mean it works as a link, but it's not what you meant to do I don't think. Doesn't show anything
@nashant It's been a while since I've worked on this but iirc because of the ugly way Docker plugins work the image isn't the plugin, you need to use the targets in the Makefile that extract the contents of that intermediary image and install the result as a plugin.
Yeah I get that it's not quite the same, but if make debug
brings up the container and I can query the API then the plugin should work. The error message I get when trying to enable is
error net dialing plugin: dial unix /run/docker/plugins/a1b8604052fb850e6f19247f3f524a771d71042047cc649d7777c1f86dc
9d535/net-dhcp.sock: connect: connection refused
Which is the same as what I get when I run the debug container but try to query the API too quickly, making me think that it's a timing issue and a longer timeout/sleep somewhere would make it happier. I'm just not quite sure where to put that.
Sorry, https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c
Yeah, I looked at this too. This solution just looked far far nicer than having to do some manual fudgery with docker components. Have you tried it?
No, the author doesn't describe how to install that driver. So, trying to manually configure static IP's for containers
it's not a timing thing. The API took 2.7s to become responsive on the pi (enable doesn't work) and 4.1 on my laptop (enable does work)
ok, @sohojmanush, how about going a level even further down. Swap docker for podman: https://www.redhat.com/sysadmin/leasing-ips-podman
I am new to this container space, understanding podman will take some more time. But, the provided by you would have resolved my issues if it is possible in docker.
Right. It's worked and it's really simple.
thank you , will give it a try and let you know.
@nashant tried your method, podman working fine. But, found out that pods only get an IP from DHCP at the start of creation. Does CNI plugins work with docker to and is there a chance that you are going upload your arm compiled docker-net-dhcp plugin?
As far as I've found no, docker can't use CNI plugins. What do you mean by 'only get an IP from DHCP at the start of creation'? Surely you only want your running containers to have IPs? I wasn't planning on pushing the arm image, no, as podman is working perfectly
What I mean is that podman containers only got an Ip from DHCP at the start of a container creation. After that no changing IPS, so I had to choose podman's choice of IP. I wanted 192.168.88.5, but podman took 192.168.88.209. So, had to make that IP static on the router.
In which case you don't want DHCP, you want something like this:
# cat /etc/cni/net.d/99-static1.conflist
{
"cniVersion": "0.4.0",
"name": "static1",
"plugins": [
{
"type": "macvlan",
"master": "eth0",
"ipam": {
"type": "static",
"addresses": [{
"address": "172.16.0.1/24",
"gateway": "172.16.0.254"
}],
"routes": [{
"dst": "0.0.0.0/0"
}],
"dns": {
"nameservers": ["172.16.0.254"],
"domain": "home",
"search": [
"home"
]
}
}
}
]
}
Any reason you don't want to use DHCP? Don't believe in DNS?
Thank you for example.I really want DHCP , so that all my containers can have a lease from the router.
I'm not sure what the problem is then. You say you want DHCP leases from the router but you want the container to choose its own IP? DHCP doesn't work like that. If you want to configure static DHCP leases then you configure the router to give a certain IP to a certain MAC address, which you say you've done. What do you actually want to occur?
I created a pihole pod, which took the Ip in my DHCP as 192.168.88.210. So, I configured my router to give the container a static IP 192.168.88.5. My router did assign that IP to the container, but the container didn't release the IP 192.168.88.210. Also, is the CNI plugin only assign a IP to a single container or can I get lease for multiple containers ?
Right, ok! If it's for pihole I would strongly recommend using a static IP rather than DHCP. You don't really want any chance of your DNS servers getting the wrong IP. Did you delete the pod and restart your router after setting the static lease? You can use the CNI network on as many containers as you like.
Thank you, pihole running okay, but having some issue assigning IP's to other containers.
Hey. I'm trying to install this on a rpi3b+ but failing. If I try to
docker plugin install
I getIf I build from source I get
Any ideas?