ministryofjustice / cloud-platform

Documentation on the MoJ cloud platform
MIT License
87 stars 44 forks source link

Add in support for snapshots to opensearch module #4678

Closed petergphillips closed 1 year ago

petergphillips commented 1 year ago

Service name

hmpps-prisoner-search

Service environment

Impact on the service

We are transitioning over from elasticsearch to opensearch and thus are looking for equivalent functionality in the opensearch module. However it appears that there is no current support for snapshots.

Copied from slack - https://mojdt.slack.com/archives/C57UPMZLY/p1690885695297769 Hi, we’ve got a new opensearch domain that we’ve been experimenting with. With our existing elasticsearch domain we have automated jobs that take regular snapshots and once a fortnight we copy the production indices into pre-production. I’m looking at how we can do something similar with opensearch. A quick search of the terraform module shows that it isn’t setup to do any snapshots yet though? Reading https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshots.html#managedomains-snapshot-registerdirectory I would need to register a s3 repository in a similar way to how we have been doing it with elasticsearch. Later on in the Automating snapshots with Snapshot Management section of the document it suggests we can use Snapshot Management (SM) to automate it all even going as far as:

Prior to SM, OpenSearch Service offered a free, automated snapshot feature that’s still turned on by default. This feature sends snapshots into the service-maintained cs-* repository. To deactivate the feature, reach out to AWS Support.

We would automated regular snapshotting with then the ability to copy the snapshot to a different environment. I’m not sure where to start with all of that though, given that we can’t even register a s3 repository at present.

Contact person

Peter Phillips

jakemulley commented 1 year ago

This has been resolved with the release of OpenSearch v1.2.0.