Not passing arguments to dhcp client program

[sotiris-bos]

Hey it is me again! Hopefully for the last time!

Pipework allows us to pass optional arguments to dhcp clients like this:

./pipework --direct-phys eno1v12 3b10e50e38a7 dhclient:"-H hostnametest" In this case "-H hostname" to tell dhclient to set a custom hostname.

It is imperative we use doublequotes after dhclient: because pipework fails to pass the arguments otherwise.

Now trying to do the same with dreamcat4/pipework:latest brings us back to the doublequote issue discussed in https://github.com/dreamcat4/docker-images/issues/46 .

I am pasting logs and docker-compose files:

https://pastebin.com/EMUXS9L8

myapp:
  image: praqma/network-multitool
  environment:
      - pipework_cmd=--direct-phys eno1v12 @CONTAINER_NAME@ dhclient:"-H hostnametest"
  net: none

Suffice it to say that I also tried it without the doublequotes but it did not work, as it did not work with pipework ran straight on the host either without those doublequotes.

[dreamcat4]

Yes its an issue here because we (the entrypoint.sh) will pass the pipework cmd through unix eval before invoking pipework.

There are a couple of things you might want to try to get around this, for example:

if dhclient lets you remove the space after -H you might see if it can be made to work without any quotes at all

    - pipework_cmd=--direct-phys eno1v12 @CONTAINER_NAME@ dhclient:-Hhostnametest

2. If not then try escaping the quotes, and/or try single quotes instead of double quotes. Sometimes this method requires multiple escapes \\\", depending how many levels are being stripped away. Perhaps the cmdline is being parsed multiple times. More than once.

Here is the simplest example of that, just 1 level of \\\ escapes

    - pipework_cmd=--direct-phys eno1v12 @CONTAINER_NAME@ dhclient:\"-H hostnametest\"

Or perhaps escaping the space character instead might preserve it in addition to removing those quotes :-H\ hostname. etc. there are many variations / combinations of ways to escape. More than i can list here. Including the single quotes too.

If its like before its probably stopping parsing at the first double quote encountered. the one immediately before the -H cutting off the rest of the string

[sotiris-bos]

Nothing seems to work, I have been trying for the past two hours. I think your script encases every argument after a space in the pipework_cmd line in singlequotes. That means that there is no way I can find to add a space between -H and hostenametest without them being interpreted as different arguments.

Escaping does not work because of the singlequotes. I also tried with ${IFS} but docker-compose completely removes that.

I am at a complete loss with this one.

Edit: Here is another example:

https://pastebin.com/ZGejR2XJ

with

myapp:
  image: praqma/network-multitool
  environment:
      - pipework_cmd=--direct-phys eno1v19 @CONTAINER_NAME@ dhclient:\"-H hostnametest\"
  net: none

As you can see on lines 493, 495, 502 the pipework command looks like the one I ran directly on the host and worked. But, on line 444 you can see that the DHCP_OPTIONS is wrong. Lines 475, 496 also indicate the problem.

[dreamcat4]

What I forgot to mention earlier was that due to your earlier complaint I had added a commit to strip exra outer quotes. Since you were unfamiliar / mislead in my documentation with the way to declare strings in YAML that it does not require the double quotes.

So I have now removed that which may / may not help but will at least try to preserve the " at the very end of the line.

The other issue we have here may be that eval is being used in combination with export in order to process multiple env vars / multiple pipework_cmd_*= for the other situations where people have to use multiple invokations of pipework. This then makes it harder to change / remove with something else instead because that can lead to unwanted breakage(s) of other feature(s).

So whilst I would like to get in there are change certain things, it is somewhat with one hand tied. Would also like to try to reproduct this myself. Just not sure if i can replicate it here yet. Need to give it a try without the sr-iov stuff.

[dreamcat4]

Got another question here: Where is the -H hostname flag of dhclient program? Sorry but I cannot see it here on my ubuntu 19.10 system. And it's not specifically documented (as the -H flag) in pipework readme either.

[id:~/docker-images] master(+66/-35)+* 1 ± dhclient --help
Usage: dhclient [-4|-6] [-SNTPRI1dvrxi] [-nw] [-p <port>] [-D LL|LLT]
                [--dad-wait-time <seconds>] [--prefix-len-hint <length>]
                [--decline-wait-time <seconds>]
                [--address-prefix-len <length>]
                [-s server-addr] [-cf config-file]
                [-df duid-file] [-lf lease-file]
                [-pf pid-file] [--no-pid] [-e VAR=val]
                [-sf script-file] [interface]*
       dhclient {--version|--help|-h}

No -H, see? So I cannot test with that. Perhaps there is some other optional flag i can use instead?

[sotiris-bos]

Huh. First of all, that change did not change anything. No matter what I do, I cannot get anything after the space after the H at the pipework_cmd to show up after DHCP_OPTIONS=-H on the debug screen.

This is the output of dhclient --help on my Centos 8 host:

dhclient --help
Internet Systems Consortium DHCP Client 4.3.6
Copyright 2004-2017 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Unknown command: --help
Usage: dhclient [-4|-6] [-SNTPRI1dvrxi] [-nw] [-p <port>]
                [-D LL|LLT] [--dad-wait-time seconds]
                [-s server-addr] [-cf config-file]
                [-df duid-file] [-lf lease-file]
                [-pf pid-file] [--no-pid] [-e VAR=val]
                [-sf script-file] [interface]*
                [-C <dhcp-client-identifier>] [-B]
                [-H <host-name> | -F <fqdn.fqdn>] [--timeout <timeout>]
                [-V <vendor-class-identifier>]
                [--request-options <request option list>]

This version of ISC DHCP is based on the release available
on ftp.isc.org. Features have been added and other changes
have been made to the base software release in order to make
it work better with this distribution.

Please report issues with this software via: 
https://bugs.centos.org/

exiting.

Clearly this is not a standard dhclient version, hence the -H option.

I will try with the other DHCP clients but if I cannot pass anything after a space then I might have to resort to modifying the pipework script and forking your image to suite my needs.

You don't need an SR-IOV interface to do this. Any unused physical interface on your machine will do. It will work better if it does not have an IP address assigned but it needs to be up (shown under ifconfig). SR-IOV is just an "easy" way to create virtual interfaces that behave like physical ones and let the NIC handle everything virtual related instead of the kernel.

Edit: I am now trying with udhcpc that needs even more spaces:

myapp:
  image: praqma/network-multitool
  environment:
      - pipework_cmd=--direct-phys eno1v19 @CONTAINER_NAME@ udhcpc:\"-i eno1v19 -x hostname:networkmulti\"
  net: none

https://pastebin.com/ySrSdALa As you can see in line 19, it only passes "-i as DHCP_OPTIONS

Edit2: On a sidenote, running:

docker run --privileged --pid=host --net=host   -v /var/run/docker.sock:/docker.sock   -e run_mode=batch,daemon   -e host_routes=true   -e route_add_delay=1  -e debug=true dreamcat4/pipework:latest

I think just pulls the latest image from dockerhub, which does not contain the changes. I am manually replacing the entrypoint.sh after starting the container.

Edit3: Oh god. Just running:

myapp:
  image: praqma/network-multitool
  environment:
      - pipework_cmd=--direct-phys eno1v19 @CONTAINER_NAME@ udhcpc
  net: none

makes the hostname the same as the container name. At least I found a way to assign hostnames after so many hours!

The not-passing-dhcp-client-arguments problem still stands. But at least as far as the hostname is concerned, pipework with udhcpc assigns it automatically as the container name.

[dreamcat4]

Yeah sorry I seem to be having a local issue here actually building and testing that latest commit change. Not sure why, it keeps using the previous docker image instead. Despite having build / pulled the latest from dockerhub. No idea why....

But what your very last comment is reminding me is about udhcpc being the best program to use instead of dhclient... pretty sure I discovered that myself also several years ago and then completely forgot about it since then. You should really just use udhcpc wherever possible. This is best overall advice when choosing between these multiple dhcp clients.

In terms of the arguments handling - yeah I agree it's probably not enough to work. The main issue seem to be that \ escaping space in a YAML file does not ever work because *for most imlpementations of the yaml interpreter), that is then always interpreted as 2 chars a real \ chracter then followed by a real space character.

Tried some other yaml space escape codes but unfortunately those did not work either. There are like these \000 unicode type ones and html type > looking ones. Neither syntax worked for me over here.

In terms of main pipework upstream script:

It's arguments handling never was so great to begin with and over time certain extra flags have just been added on piecemeal. No control over that area myself (and it would break existing users anyhow) so yes... you can just checkout and then edit my Dockerfile and use the line(s) where it says "Uncomment to hack a local copy of the pipework script". Although perhaps in this case it's fortunately not necessary anymore. Having by-now found udhcpc instead.

I would recommend we don't try continue with this right now, and unless you really need to come back to this matter again later on. Leaving this issue open (forever probably).

[sotiris-bos]

Yeah I tried with yaml escape codes as well but I had no luck.

Pipework is very useful for me who wants to assign host interfaces to containers. It seems so much easier with single docker hosts to keep all IP/hostname container info together with that of the rest of the LAN on a central DHCP/DNS server rather than splitting it. Not to mention the possible performance and offloading benefits.

I don't think this kind of functionality is ever going to be implemented on docker since, with Swarm, Kubernetes etc being all the rage, assigning physical interfaces to containers that can move between hosts doesn't work. So in a way I think it is still relevant and will continue to be.

I am really glad for all the work you have put into this and for helping me achieve what I wanted. Again, thank you very much!

As far as this issue is concerned, sure arguments with spaces are not working but single arguments without spaces or even multiple ones should work, since it seems that the pipework script handled the two parts separated by the space as separate arguments. I haven't tested it yet. I agree that we should not continue pursuing this, at least for the foreseeable future (it is an upstream issue after all) and should probably leave the issue open for anyone that might stumble upon this.

Cheers!