Open TonyNguyenAdNet opened 9 months ago
@TonyNguyenAdNet, we recently experienced the same issues running HAProxy v2.8.5 and Dataplaneapi v2.8.7 on OEL9. It looks like adding --disable-inotify
to the command in the program
section of your HAProxy config might resolve the issue. So, ours looks something like this:
program api
command dataplaneapi -f /etc/haproxy/dataplaneapi.yml --disable-inotify
no option start-on-reload
We no longer get the has at the top of the HAProxy config file and it did not re-arrange anything.
EDIT: This can be added as a config option in the yml as disable_inotify: true
I have this same issue. The --disable-inotify
solution above did not work.
I'm on haproxy 3.0.3
Im following the exact instructions from here in a Kubernetes pod: https://www.haproxy.com/documentation/haproxy-data-plane-api/installation/install-on-haproxy/#run-the-api-in-the-current-terminal-session
I'm mounting in a file:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: dataplane-conf
annotations:
argocd.argoproj.io/sync-wave: '-1'
data:
dataplaneapi.yaml: |
dataplaneapi:
disable_inotify: true
host: 0.0.0.0
port: 5555
transaction:
transaction_dir: /tmp/haproxy
user:
- insecure: false
password: xxx
name: admin
haproxy:
config_file: /usr/local/etc/haproxy/haproxy.cfg
haproxy_bin: /usr/sbin/haproxy
reload:
reload_delay: 5
reload_cmd: service haproxy reload
restart_cmd: service haproxy restart
and in the main haproxy.cfg I have:
program api
command /opt/bitnami/haproxy-dataplaneapi/bin/dataplaneapi -f /usr/local/etc/haproxy-dataplane/dataplaneapi.yaml --disable-inotify
but:
[NOTICE] (1) : New program 'api' (8) forked
[NOTICE] (1) : New worker (9) forked
[NOTICE] (1) : Loading success.
time="2024-07-30T11:10:51Z" level=fatal msg="The configuration file is not declared in the HAPROXY_CFGFILES environment variable, cannot start."
[NOTICE] (1) : haproxy version is 3.0.3-95a607c
[NOTICE] (1) : path to executable is /opt/bitnami/haproxy/sbin/haproxy
[ALERT] (1) : Current program 'api' (8) exited with code 1 (Exit)
[ALERT] (1) : exit-on-failure: killing every processes with SIGTERM
[ALERT] (1) : Current worker (9) exited with code 143 (Terminated)
[WARNING] (1) : All workers exited. Exiting... (1)
I'm going to try setting this ENV but like the posters said above,
haproxy:
config_file: /usr/local/etc/haproxy/haproxy.cfg
seems to just not work.
@tommyjcarpenter my situation involved setting dataplaneapi back to watching the actual HAProxy configuration file, as well as setting the disable_inotify
setting to true
.
@KiyoIchikawa can you please elaborate on the first part? I tried the disable_inotify
and it didnt work.
im doing this in kubernetes, which there are no instructions for, but I thought it should be the same using the program/master worker
solution above.
Im currently giving a shot at the sidecar approach instead.
So, I've realized only now that while my sidecar loads and does reads fine, it cant restart the haproxy process, because its in a separate container 😓 this should have been obvious... 20/20 hindsight.
So i'd like to circle back here, because I have tried a million ways to solve this and I keep hitting this issue.
Im trying to run the dataplaneapi in the same container as haproxy.
In my haproxy config, I have
program echo1
command cat /etc/haproxy/dataplaneapi.yaml
program echo2
command cat /etc/haproxy/haproxy.cfg
program api
command dataplaneapi -f /etc/haproxy/dataplaneapi.yaml
no option start-on-reload
both of those echo
(for debugging) print out fine.
my dataplaneapi.yaml looks like:
dataplaneapi:
log_level: debug
host: 0.0.0.0
port: 5555
transaction:
transaction_dir: /tmp/haproxy
user:
- insecure: false
password: $5$qyiFlUI2hW4JRlpv$gPAZo98q.t0ItrYZ6wyFJxfpw0n6k2VUcBUVgwt3Vj1
name: admin
haproxy:
haproxy_bin: /opt/bitnami/haproxy/sbin/haproxy
# config_file is the default
reload:
reload_delay: 5
reload_cmd: service haproxy reload
restart_cmd: service haproxy restart
I have tried /usr/etc/local/haproxy
etc, Ive tried specifying the config (im using the default now), Ive tried explicitly setting HAPROXY_CFGFILES=...
but all to no avail.
[NOTICE] (1) : haproxy version is 3.0.3-95a607c
[NOTICE] (1) : path to executable is /opt/bitnami/haproxy/sbin/haproxy
[ALERT] (1) : Current program 'echo1' (8) exited with code 0 (Exit)
[ALERT] (1) : Current program 'echo2' (9) exited with code 0 (Exit)
time="2024-07-31T19:36:57Z" level=fatal msg="The configuration file is not declared in the HAPROXY_CFGFILES environment variable, cannot start."
[ALERT] (1) : Current program 'api' (10) exited with code 1 (Exit)
[ALERT] (1) : exit-on-failure: killing every processes with SIGTERM
[ALERT] (1) : Current worker (11) exited with code 143 (Terminated)
[WARNING] (1) : All workers exited. Exiting... (1)
Maybe related, but HAProxy is mixing things up;
When executing dataplaneapi version
I get;
configuration file /etc/haproxy/dataplaneapi.yaml does not exists, creating one
configuration error: the custom reload strategy requires these options to be set: ReloadCmd, RestartCmd
and /etc/haproxy/dataplaneapi.yaml never gets created.
Thing is, dataplaneapi config is stored in /etc/haproxy/dataplaneapi.yml , not /etc/haproxy/dataplaneapi.yaml And should be stored in /etc/haproxy/dataplaneapi.yml according to their own documentation
Hi Everyone, hoping for some help!
High level goal:
Configure HAProxy via split config files, using
-f conf.d/
, similar to OS service configurations*.d/
pattern, while using dataplaneapi (latest 2.8 version) for basic configuration such as draining/enabling a backend server via remote scripting [1]. We are currently running into the issue where dataplaneapi invoked via program/command errors out due to what it believes to be a missing configuration file:time="2023-11-25T17:19:21-08:00" level=fatal msg="The configuration file is not declared in the HAPROXY_CFGFILES environment variable, cannot start."
... however the initial HAProxy config file is specified within the .yml file as so:
We have also tried the command-line argument, which results in the same error:
--config-file /f/haproxy-h3-9200/conf.d/00-haproxy.cfg
Note that even though we are specifying the first/primary config file (
00-haproxy.cfg
) to dataplaneapi, when starting dataplaneapi independently of HAProxy (ie, not usingprogram+command
), the desired management functionality works for the backend specified in the second file (01-haproxy.cfg
). It's just when using HAProxy to manage the dataplaneapi process do we see the above error.Doing some testing, it appears that HAProxy is not setting a value for this environment variable when spawning dataplaneapi via the
program
directive. As a simple test, I replaced dataplanapi with a simpleecho
command as shown in the below examples. I'm testing both our current HAProxy production version 2.2.14 as well as the latest 2.8.4, and as the output shows, the value ofHAPROXY_CFGFILES
is empty when using theecho
command. Interestingly, switching toprintenv
shows this variable being set! So perhaps there's some subtle behavior of the current sub-process creation that is not passing variables with certain sub-commands?Configuration Overview:
conf.d/00-haproxy.cfg
andconf.d/01-haproxy.cfg
are the contents of a previously working single configuration file, with00-haproxy.cfg
containing thedefaults/global
sections and01-haproxy.cfg
containing a singlelisten
directive. Setting aside dataplaneapi, the HAProxy functionality itself works fine with this configuration: the single listen configuration forwards incoming requests to the appropriate backend servers as expected.program
config:command echo "HAPROXY_CFGFILES: ${HAPROXY_CFGFILES}, PATH: ${PATH}, HAPROXY_SERVER_NAME: ${HAPROXY_SERVER_NAME}, HAPROXY_LOCALPEER: ${HAPROXY_LOCALPEER}"
With 2.2.14:
HAProxy invoke command1 (multiple config files specified):
/f/haproxy-9200/bin/haproxy-9200 -Ws -f /f/haproxy-9200/conf.d/00-haproxy.cfg -f /f/haproxy-9200/conf.d/01-haproxy.cfg -p /run/haproxy-9200.pid -S /run/haproxy-9200-master.sock
HAProxy invoke command2 (config directory specified):
/f/haproxy-9200/bin/haproxy-9200 -Ws -f /f/haproxy-9200/conf.d -p /run/haproxy-9200.pid -S /run/haproxy-9200-master.sock
Output is the same for both invocations:
With 2.8.4:
HAProxy invoke command1 (multiple config files specified):
/f/haproxy-h3-9200/bin/haproxy-h3-9200 -Ws -f /f/haproxy-h3-9200/conf.d/00-haproxy.cfg -f /f/haproxy-h3-9200/conf.d/01-haproxy.cfg -p /run/haproxy-h3-9200.pid -S /run/haproxy-h3-9200-master.sock
HAProxy invoke command2 (config directory specified):
/f/haproxy-h3-9200/bin/haproxy-h3-9200 -Ws -f /f/haproxy-h3-9200/conf.d -p /run/haproxy-h3-9200.pid -S /run/haproxy-h3-9200-master.sock
Output is the same for both invocations:
Finally, with
command printenv
I see the following output on both HAProxy versions (the variable is set):Any insight here is appreciated! We are very close to having our preferred configuration setup that is a balance between previous management styles and utilizing the latest dataplaneapi functionality.
[1] it is understood that the developers of dataplaneapi want to move administration completely under dataplanapi's control, ie, re-writing configuration files; however our production practices need to move in a slower fashion utilizing our current Ansible/Puppet management style which relies on these tools to manage config files. We will ultimately embrace dataplaneapi for management completely but now is not that time.