MSD-LIVE / issues

0 stars 0 forks source link

Do required RDM RDS for Postgres DB upgrade #217

Open zguillen opened 2 months ago

zguillen commented 2 months ago

AWS is requiring our RDS be upgraded (see comment from email sent by them 8/29). We need to first upgrade our local container's version and test locally, then see what version the new RDM we'll be upgrading to will support. Once we confirm everything is working we can do Matt's plan of upgrading in aws (see comment copied from 8/29 email with him)

zguillen commented 2 months ago

Hello,

[AWS Health may periodically trigger reminder notifications about this communication if resources remain unresolved.]

You are receiving this message because you have one or more Amazon RDS for PostgreSQL DB instances running major version 12 that require your attention. PostgreSQL 12 will be deprecated by the community in November 2024 [1], and will not receive any bug fixes or security patches after that. Following the community timeline, PostgreSQL 12 will reach RDS end of standard support on February 28, 2025. However, if you need to continue using PostgreSQL 12 beyond that date, it will be available on RDS under Amazon RDS Extended Support [2]. With Extended Support, your monthly charges will increase, but you will have time up to three years, to upgrade to a new major version. RDS will provide critical security patches and bug fixes for your RDS instance as well as the PostgreSQL database engine during that time.

If you plan to use PostgreSQL 12 beyond February 28, 2025, you can estimate your Extended Support charges by using the AWS Pricing Calculator [3]. If you want to avoid paying for RDS Extended Support, please upgrade your RDS for PostgreSQL 12 DB instance before March 1, 2025.

We recommend that you take action and upgrade your RDS for PostgreSQL 12 database (DB) instances to a higher major version, e.g., PostgreSQL 16 (or higher) before end of standard support on February 28, 2025. Amazon RDS for PostgreSQL 16 [4] includes support for logical decoding on read replicas, logical replication from standbys, and over 90 PostgreSQL extensions such as pgactive, pgvector, pg_tle, h3-pg, pg_cron, and rdkit. PostgreSQL 16 introduces a number of performance and visibility improvements including greater query parallelism, SIMD CPU acceleration, and a ‘pg_stat_io’ view that provides statistics on I/O usage. Further, with PostgreSQL 16, developers can now use SQL/JSON constructors and identity functions.

Your RDS PostgreSQL 12 instances that are enabled for extended support are listed in "Affected resources" tab of the AWS Health Dashboard.

If you have any questions or concerns, the AWS Support Team is available on AWS re:Post [5] and via AWS Support [6].

[1] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.postgresql.org%2Fsupport%2Fversioning%2F&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098455044%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=adGuMGSu21mi%2BfShWUAjosVvcFZ7e%2BtFXSaDoWy3rMA%3D&reserved=0 [2] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.aws.amazon.com%2FAmazonRDS%2Flatest%2FUserGuide%2Fextended-support.html&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098463955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=8fxbkBpdAh%2BhC6tYZfR8l6zhJvUEi5K%2FEWKLHXAq390%3D&reserved=0 [3] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalculator.aws%2F%23%2FcreateCalculator%2FRDSPostgreSQL&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098470262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=vHF10TXDit0qSFJLMUfmj1p6JObiCnPduNtGwRTiLoo%3D&reserved=0 [4] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fabout-aws%2Fwhats-new%2F2023%2F11%2Famazon-rds-postgresql-major-version-16%2F&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098474813%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=pduuxv1c1FyvvcbZ52qI0IAGakJ5s55iEaDvMlpFlls%3D&reserved=0 [5] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepost.aws%2F&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098479292%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=%2FegU8Xjpy4SlgTRZX4Z%2BdoOptuC967mCCD5Zh3Yk1S0%3D&reserved=0 [6] https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Faws.amazon.com%2Fsupport&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098483576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=hNnZecxYWn6KmuJD7RdvcuSwVYKS1U3W0X954x8GmPg%3D&reserved=0

Sincerely, Amazon Web Services

Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210


Reference: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhealth.aws.amazon.com%2Fhealth%2Fhome%3Fregion%3Dus-east-1%23%2Fevent-log%3FeventID%3Darn%3Aaws%3Ahealth%3Aus-west-2%3A%3Aevent%2FRDS%2FAWS_RDS_PLANNED_LIFECYCLE_EVENT%2FAWS_RDS_PLANNED_LIFECYCLE_EVENT_a70989ebad89bc81bff4ba348be6ad838b2422b9047136b871fb4ea9e9b926c0%26amp%3BeventTab%3Ddetails&data=05%7C02%7CCasey.Burleyson%40pnnl.gov%7Ced18a211655547f3133608dcc81bd717%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C638605269098487833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=aPITOcn0ySmxEj9JslGJ0jwCEsfz9v1s42n6cGdZbqU%3D&reserved=0

zguillen commented 2 months ago

From: Macduff, Matthew C Matt.Macduff@pnnl.gov Sent: Thursday, August 29, 2024 9:39 AM To: Mcallester, Devin S devin.mcallester@pnnl.gov; Burleyson, Casey D Casey.Burleyson@pnnl.gov; Lansing, Carina S Carina.Lansing@pnnl.gov; Guillen, Zoe C zoe@pnnl.gov Subject: RE: [Action required to avoid increase in charges] Amazon RDS for PostgreSQL 12 reaching end of standard support on February 28, 2025. Upgrade before March 1, 2025 to avoid paying Extended Support fees [AWS Account: 889772541283] [US-WEST-2]

Btw. May need to update codebuild base image to v5. This may be no big deal since we just do docker build….

-matt

From: Macduff, Matthew C Sent: Thursday, August 29, 2024 9:28 AM To: Mcallester, Devin S [devin.mcallester@pnnl.gov](mailto:devin.mcallester@pnnl.gov); Burleyson, Casey D [Casey.Burleyson@pnnl.gov](mailto:Casey.Burleyson@pnnl.gov); Lansing, Carina S [Carina.Lansing@pnnl.gov](mailto:Carina.Lansing@pnnl.gov); Guillen, Zoe C [zoe@pnnl.gov](mailto:zoe@pnnl.gov) Subject: RE: [Action required to avoid increase in charges] Amazon RDS for PostgreSQL 12 reaching end of standard support on February 28, 2025. Upgrade before March 1, 2025 to avoid paying Extended Support fees [AWS Account: 889772541283] [US-WEST-2]

Too many choices😊
My preference is to snapshot the database and the click on modify and pick new version.

Unless you have particular postgres special features you used, it should work without problem.

The cloud formation way is to snapshot, destroy, redeploy new based on snapshot.

-matt

From: Mcallester, Devin S [devin.mcallester@pnnl.gov](mailto:devin.mcallester@pnnl.gov) Sent: Thursday, August 29, 2024 8:31 AM To: Burleyson, Casey D [Casey.Burleyson@pnnl.gov](mailto:Casey.Burleyson@pnnl.gov); Lansing, Carina S [Carina.Lansing@pnnl.gov](mailto:Carina.Lansing@pnnl.gov); Guillen, Zoe C [zoe@pnnl.gov](mailto:zoe@pnnl.gov); Macduff, Matthew C [Matt.Macduff@pnnl.gov](mailto:Matt.Macduff@pnnl.gov) Subject: Re: [Action required to avoid increase in charges] Amazon RDS for PostgreSQL 12 reaching end of standard support on February 28, 2025. Upgrade before March 1, 2025 to avoid paying Extended Support fees [AWS Account: 889772541283] [US-WEST-2]

Yes. Our RDM database is using PostgreSQL 12.17. It will need to be updated to a newer version.

I've included Matt in case he has any suggestions for how best to update. We can discuss this more at our meeting today.

zguillen commented 2 months ago

Another RDM deployment upgraded and noted this process, here's the discord thread:

https://discord.com/channels/692989811736182844/1224742258415239269/1278662312248610849

TUW just upgraded the inveniordm instances to run postgres v16 the procedure was pretty simple [1]: 1) create SQL dump from v12 2) update the passwords in the dump 3) remove DB container volume 4) update Dockerifle/docker-compose to use v16 5) start the new db container to init the volume 6) restore the updated SQL dump from v12

[1] the only obstacle was a change in the hashing algorithm for the database users

@Sotiris performed the upgrade, so he has more insights

1 [3:28 AM] maybe you could also update the DB users' passwords after the SQL dump restore, but i'm not 100% sure [3:30 AM] also, it's noteworthy that we have configured volumes for our containers (as that's not the case in the default cookiecutter), so that the data survives a docker compose down – we had to docker volume rm in step 3