yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.02k stars 1.08k forks source link

[YSQL] Collect InitDB errors in log file #24813

Open sushantrmishra opened 2 weeks ago

sushantrmishra commented 2 weeks ago

Jira Link: DB-13919

Description

Because initdb often emits errors to stderr, and because much of our infrastructure ignores stderr, modify pg_wrapper.cc to collect stderr instead of depending on other infra. Then our existing logs should have the stderr output of initdb.

Do the same for the postgres process as well, which emits to stderr before it hooks up logging. An alternative implementation for the postgres process would be to detect when the process has a failure exit code and if it does, output stderr, or some limited tail snippet of it (but at least a few pages I would think).

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

foucher commented 1 week ago

1) The initdb part was implemented in f86bd6cff3215f7b5f8d16865fe79fb5c90ac2c0, in pg_wrapper.cc:PgWrapper::InitDb. This should be straightforward to backport. The value would probably be for local initdb failures because I'm guessing most or all actual installations use the catalog snapshot instead of running global initdb (prior to the live major ysql catalog upgrade functionality, that is). 2) The postgres part may be doable but doesn't seem to be as simple as the change to collect initdb output. Since the postgres postmaster process is long-lived, we can't use the Call interface as we do for initdb. There isn't a one-shot function in class Subprocess to start a process and wire up file descriptors. But class Subprocess seems to allow collecting of stdout and stderr so it looks to be possible. But there is also a question of how to collect stdout and stderr from the processes spawned by the postmaster.