search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.93k stars 1.06k forks source link

[YSQL] Cluster wide postgres configuration #1140

Open rao-vasireddy opened 5 years ago

rao-vasireddy commented 5 years ago

Jira Link: DB-1123 Currently postgres configuration exists on each node in postgresql.conf file. We need to provide a way to have cluster wide configuration for postgres.

rao-vasireddy commented 5 years ago

a. Alter system b. yugaware support

srhickma commented 5 years ago

For RBAC (#869), this type of cluster-wide configuration will also be needed for the pg_hba.conf and pg_ident.conf configuration files.

mrpanda commented 5 years ago

I have been thinking about this -- it is a very difficult concept. However, I think the terminology of "Master" is making it difficult, because there are many "masters" -- but as far as config files are concerned -- there is only ONE true Master - the Master of Masters. I will refer to that as MoM.

A cluster is started from the MoM, that now becomes the source of truth for initiation. Whilst the config can be loaded into a shared repository source for reference by all nodes/pods (for example, pg_catalog), the issue of those same settings being continued if the entire cluster dies and is started from a new MoM.

Does the configuration of the prior Mom take precedence? Or do we rely on the intelligence of the human administrators to ensure that any common config files are synchronized and if appropriate, make changes that would apply if that node/pod was then promoted to be the new MoM?

I hope I'm making sense. My thoughts on this are that it should be the responsibility of the humans to co-ordinate the files because there might be a genuine need for differences in the files depending upon the circumstances. Maybe the last settings are used and when a universe is restarted and new settings are used that this is logged as a warning that can be monitored so that awareness is raised ?

In conclusion I am saying that everything should not be automated as it may introduce new issues. Therefore, effort should be made in documenting the steps that admins should follow to ensure config files are synchronized where appropriate (as an option only, not a mandatory requirement -- and therefore is not the responsibility of YB to perform) and consider persisting the configs in the database (if they are not already persisted) -- and ensure that any code does not try to reach out to a file but instead uses the persisted values, and that startup of a universe compares the settings to the persisted prior settings and logs a warning to alert admins in case that represents a mistake. YB could also have a new startup flag that aborts if there is a difference or to ignore or warn.

Anyway, that is my thoughts. I welcome any comments and differences of opinion :)

ndeodhar commented 5 years ago

This is great input @mrpanda, thank you! We haven't yet finalized on the approach that we'll use but some of the thoughts that we've had on handling this include (and this overlaps with some of your suggestions):

  1. Get rid of config files and instead use system tables on master for configurations. Any changes to those will be done via commands/QL.
  2. Continue using config files but sync the changes across nodes. In case of conflicts, we'll use last writer wins type of semantics.
Bessonov commented 5 years ago

May be interesting: https://www.cockroachlabs.com/docs/stable/set-cluster-setting.html