search

devplayer0 / docker-net-dhcp

Docker network driver for networking on a host bridge with DHCP-allocated IP addresses
GNU General Public License v3.0
193 stars 55 forks source link

Unable to install plugin #10

Closed nashant closed 3 years ago

nashant commented 3 years ago

Hey. I'm trying to install this on a rpi3b+ but failing. If I try to docker plugin install I get

Error response from daemon: dial unix /run/docker/plugins/f93f40fae3f3c9431dc6fdd279b2542a9c81e42a3d0c8703a15e538af2aed01c/net-dhcp.sock: connect: no such file or directory

If I build from source I get

Error response from daemon: dial unix /run/docker/plugins/1634f7b57818e5f8258a603efb05c6891577e597107d25dd20ee6a14e62009da/net-dhcp.sock: connect: connection refused

Any ideas?

sohojmanush commented 3 years ago

Same issue: `pi@raspberrypi:~ $ sudo docker plugin ls ID NAME DESCRIPTION ENABLED c9ff714bd90b devplayer0/net-dhcp:latest Docker host bridge DHCP networking false

pi@raspberrypi:~ $ sudo docker plugin enable c9ff714bd90b Error response from daemon: dial unix /run/docker/plugins/c9ff714bd90b282648abed6a833812ad1a6e56d5cdfa1270a80046c4acc98eae/net-dhcp.sock: connect: no such file or directory`

nashant commented 3 years ago

@sohojmanush your problem there is that the plugin is not built for ARM, you have to build from source. If you enable debug logging on your docker daemon you'll see it exec format error, meaning wrong architecture.

Having built from source my syslog with docker daemon debug logging on says this:

Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.984271891Z" level=debug msg="Calling HEAD /_ping"
Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.988903534Z" level=debug msg="Calling POST /v1.41/plugins/devplayer0/net-dhcp:latest/enable?timeout=120"
Mar 29 11:38:32 pi2 dockerd[1768]: time="2021-03-29T11:38:32.995545011Z" level=debug msg="bundle dir created" bundle=/run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 module=libcontainerd namespace=plugins.moby root=/var/lib/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/rootfs
Mar 29 11:38:33 pi2 containerd[1691]: time="2021-03-29T11:38:33.168091953Z" level=info msg="starting signal loop" namespace=plugins.moby path=/run/containerd/io.containerd.runtime.v2.task/plugins.moby/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 pid=4425
Mar 29 11:38:33 pi2 systemd[1]: Started libcontainer container 0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.
Mar 29 11:38:33 pi2 systemd[1]: run-docker-runtime\x2drunc-plugins.moby-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5-runc.WqBeW5.mount: Succeeded.
Mar 29 11:38:33 pi2 systemd[3246]: run-docker-runtime\x2drunc-plugins.moby-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5-runc.WqBeW5.mount: Succeeded.
Mar 29 11:38:33 pi2 dockerd[1768]: time="2021-03-29T11:38:33.670772855Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/create
Mar 29 11:38:33 pi2 dockerd[1768]: time="2021-03-29T11:38:33.783955044Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/start
Mar 29 11:38:38 pi2 systemd-resolved[1572]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Mar 29 11:38:38 pi2 systemd-resolved[1572]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
Mar 29 11:38:43 pi2 dockerd[1768]: time="2021-03-29T11:38:43Z" level=error msg=" * Running on unix:///run/docker/plugins/net-dhcp.sock (Press CTRL+C to quit)" plugin=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.289491267Z" level=debug msg="error net dialing plugin: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused"
Mar 29 11:38:46 pi2 systemd[1]: runc-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.scope: Succeeded.
Mar 29 11:38:46 pi2 systemd[1]: runc-0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5.scope: Consumed 5.418s CPU time.
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.515876811Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/exit
Mar 29 11:38:46 pi2 containerd[1691]: time="2021-03-29T11:38:46.651446243Z" level=info msg="shim disconnected" id=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5
Mar 29 11:38:46 pi2 containerd[1691]: time="2021-03-29T11:38:46.651963637Z" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.651788325Z" level=debug msg=event module=libcontainerd namespace=plugins.moby topic=/tasks/delete
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.651977178Z" level=info msg="ignoring event" container=0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5 module=libcontainerd namespace=plugins.moby topic=/tasks/delete type="*events.TaskDelete"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698413146Z" level=debug msg="Clean shutdown of plugin"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698807779Z" level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused" error_type="*net.OpError" module=api
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.698996164Z" level=error msg="Handler for POST /v1.41/plugins/devplayer0/net-dhcp:latest/enable returned error: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused"
Mar 29 11:38:46 pi2 dockerd[1768]: time="2021-03-29T11:38:46.699351943Z" level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: dial unix /run/docker/plugins/0b821db5acdf809fa579094c814e41349fe32bb5aefec2636db2a0d3f7bdbaa5/net-dhcp.sock: connect: connection refused" error_type="*net.OpError" module=api

I can see some DNS requests being made by the plugin for /.home on my DNS server (.home being my domain). That's never gonna work and I'm not sure why it would be doing that, but I don't think that's the issue because the same happens for amd64 machines and they work fine.

sohojmanush commented 3 years ago

@nashant , thank you for clarifying I was suspecting the same.

devplayer0 commented 3 years ago

Yep, this is an issue with the plugin being currently x86_64 only. Unfortunately I do not have the time to work on this plugin at the moment, my plan is to rewrite the whole thing in a month or so when I graduate from university :)

nashant commented 3 years ago

So I get why the install won't work, but the build from source should be fine because python:3-alpine has an arm64 image, no? Could you tell me how I can get the debug logs for the plugin itself? I'm happy to create a patch

nashant commented 3 years ago

Also, if I docker run --rm --name netdhcp -v /run/docker/plugins:/run/docker/plugins devplayer0/net-dhcp:rootfs then it works. I can then

# curl -XPOST --unix-socket /run/docker/plugins/net-dhcp.sock http://localhost/NetworkDriver.GetCapabilities
{"ConnectivityScope":"global","Scope":"local"}

And if I manage to get to the socket in /run/docker/plugins/<hash>/net-dhcp.sock quickly enough then it works too. Could there be a timeout thing where it just takes longer to get running on a pi?

sohojmanush commented 3 years ago

Here is another one https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c

nashant commented 3 years ago

Here is another one https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c

That link doesn't work Edit: I mean it works as a link, but it's not what you meant to do I don't think. Doesn't show anything

sohojmanush commented 3 years ago

Sorry, https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c

devplayer0 commented 3 years ago

@nashant It's been a while since I've worked on this but iirc because of the ugly way Docker plugins work the image isn't the plugin, you need to use the targets in the Makefile that extract the contents of that intermediary image and install the result as a plugin.

nashant commented 3 years ago

Yeah I get that it's not quite the same, but if make debug brings up the container and I can query the API then the plugin should work. The error message I get when trying to enable is

error net dialing plugin: dial unix /run/docker/plugins/a1b8604052fb850e6f19247f3f524a771d71042047cc649d7777c1f86dc
9d535/net-dhcp.sock: connect: connection refused

Which is the same as what I get when I run the debug container but try to query the API too quickly, making me think that it's a timing issue and a longer timeout/sleep somewhere would make it happier. I'm just not quite sure where to put that.

nashant commented 3 years ago

Sorry, https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c

Yeah, I looked at this too. This solution just looked far far nicer than having to do some manual fudgery with docker components. Have you tried it?

sohojmanush commented 3 years ago

No, the author doesn't describe how to install that driver. So, trying to manually configure static IP's for containers

nashant commented 3 years ago

it's not a timing thing. The API took 2.7s to become responsive on the pi (enable doesn't work) and 4.1 on my laptop (enable does work)

nashant commented 3 years ago

ok, @sohojmanush, how about going a level even further down. Swap docker for podman: https://www.redhat.com/sysadmin/leasing-ips-podman

sohojmanush commented 3 years ago

I am new to this container space, understanding podman will take some more time. But, the provided by you would have resolved my issues if it is possible in docker.

nashant commented 3 years ago

Right. It's worked and it's really simple.

  1. Install podman. I'm on ubuntu 20.04 and used https://www.hostnextra.com/kb/easy-to-install-podman-on-ubuntu-20-04/
  2. Install podman-compose. https://fedoramagazine.org/manage-containers-with-podman-compose/
  3. Download latest tarball from https://github.com/containernetworking/plugins/releases and extract to /usr/libexec/cni.
  4. Create cni config file and systemd units from https://www.redhat.com/sysadmin/leasing-ips-podman
  5. Profit
sohojmanush commented 3 years ago

thank you , will give it a try and let you know.

sohojmanush commented 3 years ago

@nashant tried your method, podman working fine. But, found out that pods only get an IP from DHCP at the start of creation. Does CNI plugins work with docker to and is there a chance that you are going upload your arm compiled docker-net-dhcp plugin?

nashant commented 3 years ago

As far as I've found no, docker can't use CNI plugins. What do you mean by 'only get an IP from DHCP at the start of creation'? Surely you only want your running containers to have IPs? I wasn't planning on pushing the arm image, no, as podman is working perfectly

sohojmanush commented 3 years ago

What I mean is that podman containers only got an Ip from DHCP at the start of a container creation. After that no changing IPS, so I had to choose podman's choice of IP. I wanted 192.168.88.5, but podman took 192.168.88.209. So, had to make that IP static on the router.

nashant commented 3 years ago

In which case you don't want DHCP, you want something like this:

# cat /etc/cni/net.d/99-static1.conflist 
{
    "cniVersion": "0.4.0",
    "name": "static1",
    "plugins": [
        {
            "type": "macvlan",
            "master": "eth0",
            "ipam": {
                "type": "static",
                "addresses": [{
                    "address": "172.16.0.1/24",
                    "gateway": "172.16.0.254"
                }],
                "routes": [{
                    "dst": "0.0.0.0/0"
                }],
                "dns": {
                    "nameservers": ["172.16.0.254"],
                    "domain": "home",
                    "search": [
                      "home"
                    ]
                }
            }
        }
    ]
}

Any reason you don't want to use DHCP? Don't believe in DNS?

sohojmanush commented 3 years ago

Thank you for example.I really want DHCP , so that all my containers can have a lease from the router.

nashant commented 3 years ago

I'm not sure what the problem is then. You say you want DHCP leases from the router but you want the container to choose its own IP? DHCP doesn't work like that. If you want to configure static DHCP leases then you configure the router to give a certain IP to a certain MAC address, which you say you've done. What do you actually want to occur?

sohojmanush commented 3 years ago

I created a pihole pod, which took the Ip in my DHCP as 192.168.88.210. So, I configured my router to give the container a static IP 192.168.88.5. My router did assign that IP to the container, but the container didn't release the IP 192.168.88.210. Also, is the CNI plugin only assign a IP to a single container or can I get lease for multiple containers ?

nashant commented 3 years ago

Right, ok! If it's for pihole I would strongly recommend using a static IP rather than DHCP. You don't really want any chance of your DNS servers getting the wrong IP. Did you delete the pod and restart your router after setting the static lease? You can use the CNI network on as many containers as you like.

sohojmanush commented 3 years ago

Thank you, pihole running okay, but having some issue assigning IP's to other containers.