yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.04k stars 1.08k forks source link

[YSQL] Postgres crash loop seen if there are config failures in the nodes #25001

Open shishir2001-yb opened 5 days ago

shishir2001-yb commented 5 days ago

Jira Link: DB-14145

Description

Version: 2024.1.3.1-b8

When there are configuration issues, such as missing required g-flags, nodes can enter a Postgres crash loop. This was observed during a g-flag upgrade when adding the following flags:

"ysql_yb_enable_ash":"true"
"allowed_preview_flags_csv":"ysql_yb_enable_ash"

The g-flags upgrade failed because one node encountered a Postgres crash loop caused by the absence of the ysql_yb_ash_enable_infra g-flag. Ideally, the node should not enter a crash loop due to such configuration issues.

 yb-tserver[409778]: 2024-11-20 13:05:37.974 UTC [409778] LOG:  invalid value for parameter "yb_enable_ash": 1
 yb-tserver[409778]: 2024-11-20 13:05:37.974 UTC [409778] DETAIL:  ysql_yb_ash_enable_infra must be enabled.
 yb-tserver[409778]: 2024-11-20 13:05:37.974 UTC [409778] FATAL:  configuration file "/mnt/d0/pg_data/ysql_pg.conf" contains errors

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information