Citus master_add_node Authentication Failure with ~/.pgpass File

Attempting to configure a simple Citus cluster on Kubernetes (k8s) for testing purposes. The issue arises when utilizing the master_add_node command in the database. Citus fails to fetch data from the ~/.pgpass file, leading to authentication errors. The problem persists regardless of whether the .pgpass file is located in the PostgreSQL user’s home directory or another user’s directory. I can connect to a worker from the master pod using the following command:

psql --host=citus-worker-0.citus-workers --username=postgres
psql (16.2 (Debian 16.2-1.pgdg120+2))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

postgres=# \q

I can connect to a master from the master pod using the following command:

psql --host=citus-master --username=postgres
psql (16.2 (Debian 16.2-1.pgdg120+2))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

postgres=# SELECT * from master_add_node('citus-worker-0.citus-workers', 5432);
ERROR:  connection to the remote node citus-worker-0.citus-workers:5432 failed with the following error: FATAL:  password authentication failed for user "postgres"
postgres=# \q

As you can see i have successful connection, but when I'm run the master_add_node command from the master pod, specifying the worker details I have authentication failure. Expected Behavior: The master_add_node command should successfully authenticate and add the specified node to the Citus cluster.

Additional Information: Citus version: citusdata/citus:12.1.2 Contents of the .pgpass file:

cat /var/lib/postgresql/.pgpass
citus-master:5432:*:postgres:dsbhjq632GDHshq8dSH
citus-worker-0.citus-workers:5432:*:postgres:dsbhjq632GDHshq8dSH

More details can be found here (link to your documentation).

Patroni Kubernetes configurations are not desired as they configure the Citus cluster using the Kubernetes API, which is incompatible with a multi-region installation requirement.

citusdata / citus

Citus master_add_node Authentication Failure with ~/.pgpass File #7567