Renew OpenSearch reserved instance leases

[hannes-ucsc]

Apparently it is not possible to purchase a lease and pick the starting date so we need to make the purchase on the day of expiration of the existing lease.

dev December 20, 2024, 03:30 pm anvildev December 20, 2024, 03:26 pm prod January 01, 2025, 09:59 am anvilprod December 20, 2024, 03:06 pm

The dev lease should be reduced to four nodes (from six). All leases should be changed to the r7gd.xlarge.search instance type (from r6gd.xlarge.search).

• [ ] Prepare PR for the instance type and node count changes for all lower deployments
• [ ] Prepare hotfix PR for the instance type change for prod, anvilprod and hammerbox

The PR checklist should be amended to include the steps outlined below.

On December 20

• [ ] Push PR branch to GitLab dev but cancel deploy job of the sandbox pipeline
• [ ] sandbox pipeline is stopped at the deploy job
• [ ] Modify instance type for azul-index-sandbox manually (it should trigger a blue/green deployment)
• [ ] Blue/green deployment of azul-index-sandbox is complete and domain status is green
• [ ] Resume the sandbox pipeline
• [ ] sandbox pipeline is complete
• [ ] Merge the PR and push merge commit to GitLab anvildev but cancel deploy job of the anvilbox pipeline
• [ ] anvilbox pipeline is stopped at the deploy job
• [ ] Modify instance type for azul-index-anvilbox manually (it should trigger a blue/green deployment)
• [ ] Blue/green deployment of azul-index-anvilbox is complete and domain status is green
• [ ] Resume the anvilbox pipeline
• [ ] anvilbox pipeline is complete

December 21

• [ ] File backport PR for hotfix PR

TBC

[hannes-ucsc]

Spike to check if instance type and count change requires resource replacement in TF, using terraform plan or equivalent.