yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.01k stars 1.07k forks source link

-cert-file-pattern and --key-file-pattern did not work, when it was the same key file name / cert file name for all nodes #13234

Open upenbendre opened 2 years ago

upenbendre commented 2 years ago

Description

TServer config:

--tserver_master_addrs=ybc2ms01.ssnc-corp.cloud:7100, ybc2ms02.ssnc-corp.cloud:7100, ybc2ms03.ssnc-corp.cloud:7100
--fs_data_dirs=/home/svc_pgdev/mnt/d0
--rpc_bind_addresses=ybc2ms03.ssnc-corp.cloud:9100
--cql_proxy_bind_address=ybc2ms03.ssnc-corp.cloud:9042
--redis_proxy_bind_address=ybc2ms03.ssnc-corp.cloud:6379
--pgsql_proxy_bind_address=ybc2ms03.ssnc-corp.cloud:5433
--max_clock_skew_usec=5000000
--webserver_interface=ybc2ms03.ssnc-corp.cloud
--certs_dir=/home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts
--use_node_to_node_encryption=true
--server_broadcast_addresses=ybc2ms03.ssnc-corp.cloud:9100
--certs_for_client_dir=/home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts
--use_client_to_server_encryption=true
--allow_insecure_connections=true
--cert_file_pattern=node.crt
--key_file_pattern=node.key

Master config:

--master_addresses=ybc2ms01.ssnc-corp.cloud:7100, ybc2ms02.ssnc-corp.cloud:7100, ybc2ms03.ssnc-corp.cloud:7100
--fs_data_dirs=/home/svc_pgdev/mnt/d0
--rpc_bind_addresses=ybc2ms03.ssnc-corp.cloud:7100
--webserver_interface=ybc2ms03.ssnc-corp.cloud
--max_clock_skew_usec=5000000
--enable_automatic_tablet_splitting=TRUE
--certs_dir=/home/svc_pgdev/mnt/d0/yb-data/master/data/venaficerts
--use_node_to_node_encryption=true
--server_broadcast_addresses=ybc2ms03.ssnc-corp.cloud:9100
--cert_file_pattern=node.crt
--key_file_pattern=node.key

The permissions on cert files:

-rw-r--r--. 1 svc_pgdev centdef 6776 Jul 9 16:04 /home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts/ca.crt
-rw-r--r--. 1 svc_pgdev centdef 2478 Jul 9 16:04 /home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts/node.crt
-r--------. 1 svc_pgdev centdef 1679 Jul 9 16:04 /home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts/node.key

Results: Postgres 5433 port process shuts down: yb-tserver.out

2022-07-09 16:05:53.502 UTC [82698] LOG: database system is shut down
2022-07-09 16:05:53.629 UTC [82700] LOG: YugaByte is ENABLED in PostgreSQL. Transactions are enabled.
2022-07-09 16:05:53.634 UTC [82700] FATAL: could not load server certificate file “/home/svc_pgdev/mnt/d0/yb-data/tserver/data/venaficerts/node.ybc2ms02.ssnc-corp.cloud.crt”: No such file or directory

I have provided a cert file pattern and key file pattern without a $ variable, since I want to use the same crt/key files across nodes. But Yugabyte seems to ignore that and look for node..key/crt - going by the error above. I am using v2.12. Do the cert and key file pattern flags work on my version, and are they correct? If this generic file option is not supported today, it would be great if it is because if we are using a single key file for all nodes, we wish to have a simple pattern.

ddorian commented 2 years ago

cc @sushantrmishra