Simplify pageserver initialization & configuration

neondatabase / neon

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.

https://neon.tech

Apache License 2.0

15.03k stars 439 forks source link

Simplify pageserver initialization & configuration #7736

Closed VladLazar closed 3 months ago

VladLazar commented 6 months ago

Overall plan: https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

### Tasks
- [ ] https://github.com/neondatabase/aws/pull/1322
- [ ] https://github.com/neondatabase/neon/pull/7766
- [ ] https://github.com/neondatabase/cloud/pull/13417
- [x] notify #neon-oss users on Discord; https://discord.com/channels/1176467419317940276/1184894814769127464/1266346201234804827
- [ ] https://github.com/neondatabase/aws/pull/1635
- [ ] https://github.com/neondatabase/cloud/pull/15863
- [ ] https://github.com/neondatabase/cloud/pull/15874
- [ ] https://github.com/neondatabase/neon/pull/8489

VladLazar commented 5 months ago

Last week (ending 2024-05-17):

pageserver pr: https://github.com/neondatabase/neon/pull/7766
cplane pr to remove usage of config override: https://github.com/neondatabase/cloud/pull/13417
ansible pr: https://github.com/neondatabase/aws/pull/1322

Pageserver and ansible changes are good to go, but cplane usage of --config-override is blocking them. The cplane changes have to merge first. Once that's done apply the plan from here: https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea

Once Christian and/or Vlad are back from holiday:

Get cplane pr merged (doesn't 100% work in current state)
Apply rollout plan https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea

VladLazar commented 5 months ago

2024-06-12:

Discussed the removal --config-override from cplane repo with Alexey. The control plane will pick it up.
We can resume work on our side once that's happened.

VladLazar commented 4 months ago

2024-07-01:

Waiting for control plane to remove uses of --config-override flag - should be done early this week
Rebase everything and merge

VladLazar commented 4 months ago

2024-07-08:

https://github.com/neondatabase/cloud/pull/13417 merged. Can now proceed with rebasing everything and merging the aws and neon prs

problame commented 3 months ago

This epic is complete as of this week's storage deploy. https://neondb.slack.com/archives/C03H1K0PGKH/p1722347077872989?thread_ts=1722244979.635389&cid=C03H1K0PGKH