Command parsing when map id is negative

[shauntarves]

I am running my haproxy in a container with a volume-mounted socket directory, and when listing maps via the stats socket, the map id is -1. This creates a problem because the command-line parsing doesn't work (in any way that I've tried) and the actual map file is inside of the container, so the os.path.isfile check fails as well. I'm at a loss how to deal with this:

[root]# echo "show map" | socat stdio /var/lib/docker/volumes/run/_data/haproxy.sock
# id (file) description
-1 (/usr/local/etc/haproxy/domain-to-backend.map) pattern loaded from file '/usr/local/etc/haproxy/domain-to-backend.map' used by map at file '/usr/local/etc/haproxy/haproxy.cfg' line 77, by map at file '/usr/local/etc/haproxy/haproxy.cfg' line 94

[root]# echo "show map /usr/local/etc/haproxy/domain-to-backend.map" | socat stdio /var/lib/docker/volumes/run/_data/haproxy.sock
0x555fa64b4d00 xx1 yy1
0x555fa64b4dc0 xx2 yy2
0x555fa64b4ea0 xx3 yy3

[root]# echo "show map #-1" | socat stdio /var/lib/docker/volumes/run/_data/haproxy.sock
0x555fa64b4d00 xx1 yy1
0x555fa64b4dc0 xx2 yy2
0x555fa64b4ea0 xx3 yy3

[root]#  haproxytool map -F /var/lib/docker/volumes/run/_data/haproxy.sock --list
# id (file) description
-1 (/usr/local/etc/haproxy/domain-to-backend.map) pattern loaded from file '/usr/local/etc/haproxy/domain-to-backend.map' used by map at file '/usr/local/etc/haproxy/haproxy.cfg' line 77, by map at file '/usr/local/etc/haproxy/haproxy.cfg' line 94

[root]#  haproxytool map -F /var/lib/docker/volumes/run/_data/haproxy.sock -s -1
Usage:
    haproxytool map [-D DIR | -F SOCKET] -l
    haproxytool map [-D DIR | -F SOCKET] (-s | -c ) MAPID
    haproxytool map [-D DIR | -F SOCKET] -g MAPID KEY
    haproxytool map [-D DIR | -F SOCKET] (-S | -A) MAPID KEY VALUE
    haproxytool map [-D DIR | -F SOCKET] -d MAPID KEY

[root]#  haproxytool map -F /var/lib/docker/volumes/run/_data/haproxy.sock -s "-1"
Usage:
    haproxytool map [-D DIR | -F SOCKET] -l
    haproxytool map [-D DIR | -F SOCKET] (-s | -c ) MAPID
    haproxytool map [-D DIR | -F SOCKET] -g MAPID KEY
    haproxytool map [-D DIR | -F SOCKET] (-S | -A) MAPID KEY VALUE
    haproxytool map [-D DIR | -F SOCKET] -d MAPID KEY

[root]# haproxytool map -F /var/lib/docker/volumes/run/_data/haproxy.sock --show=-1
--show must not have an argument
Usage:
    haproxytool map [-D DIR | -F SOCKET] -l
    haproxytool map [-D DIR | -F SOCKET] (-s | -c ) MAPID
    haproxytool map [-D DIR | -F SOCKET] -g MAPID KEY
    haproxytool map [-D DIR | -F SOCKET] (-S | -A) MAPID KEY VALUE
    haproxytool map [-D DIR | -F SOCKET] -d MAPID KEY

[root]# haproxytool map -F /var/lib/docker/volumes/run/_data/haproxy.sock -s "/usr/local/etc/haproxy/domain-to-backend.map"
Invalid input

[unixsurfer]

I think the fact that the id of the map has "-" confuses docopt library, which parses the CLI arguments. I can reproduce the issue. But, in my case I can simply get the content of the map by using the filename, as you tried uncessfully in your last example, for the map file /etc/lb_engine/data/generated/maps/staging-hosts with id -1 the following yield the content of the map

sudo haproxytool map -D /run/lb_engine -s /etc/lb_engine/data/generated/maps/staging-hosts

or

sudo haproxytool map -D /run/lb_engine -s "/etc/lb_engine/data/generated/maps/staging-hosts"

Wh version of python and docopt do you use?

[shauntarves]

I agree that it is the fault of docopt not being able to parse the -1. That is documented elsewhere in issues in their own repository.

The second, more subtle issue of not being able to get by filename is because your code in the show_map method in haproxyadmin does the os.path.isfile test and the file being referenced is inside the container - not on the host system from where I run the haproxytool command.

If I remove that elif os.path.isfile test in the show_map method and just let it run the command, the tool returns the correct data. So, I guess I'm saying that trying to check the existence of the file there is causing a work-around for the negative number issue to also fail.

So, as such, I'm left with no way to access my data without manually adjusting the code

[unixsurfer]

Now that you mentioned container

I agree that it is the fault of docopt not being able to parse the -1. That is documented elsewhere in issues in their own repository.

The second, more subtle issue of not being able to get by filename is because your code in the show_map method in haproxyadmin does the os.path.isfile test and the file being referenced is inside the container - not on the host system from where I run the haproxytool command.

If I remove that elif os.path.isfile test in the show_map method and just let it run the command, the tool returns the correct data. So, I guess I'm saying that trying to check the existence of the file there is causing a work-around for the negative number issue to also fail.

We never get the value "-1" thus we print the usage and we can't do much about it. But, I think we simply remove the check if it is a file and pass to haproxy whatever we get. In the worst case scenario, we will return Unknown map identifier. Please use #<id> or <file>.

What do you think?

[shauntarves]

That sounds great!

[unixsurfer]

question, how come can you access the socket file but not the map file? If we skip the os.path.isfile check for files then we should skip it also for socket files. The latter will complicate a bit our error handling.

[shauntarves]

In order to have the socket file accessible from multiple peers of haproxy running as containers, it gets put into a docker shared volume that is mounted to multiple containers, but lives on the host machine.

The map file, on the other hand, is baked into our container.

We are trying to run this python management command from the host machine.

Largely, these are design decisions that could have been done differently, but this is what we have.

[shauntarves]

Another option, of course, is to use the long form of the arguments in docopt, which would allow the user to specify a map value of -1. This could be done as --show=-1. This is the suggested solution from the docopt team as well. https://github.com/docopt/docopt/issues/158

[unixsurfer]

Another option, of course, is to use the long form of the arguments in docopt, which would allow the user to specify a map value of -1. This could be done as --show=-1. This is the suggested solution from the docopt team as well. docopt/docopt#158

We currently use MAPID as an argument and if I make it an option with an argument(--mapid=-1) then we complete change CLI in an unportable way.

[unixsurfer]

You may want to read https://www.mail-archive.com/haproxy@formilux.org/msg33223.html) and http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=0f93672dfea805268d674c97573711fbff7e0e70