metacpan / metacpan-puppet

Puppet configuration for running metacpan servers
Other
21 stars 17 forks source link

Migrate ES to hosted service #240

Open ranguard opened 4 months ago

ranguard commented 4 months ago

Migrate

Olaf: I will update metacpan credentials shortly, but:

We now have a hosted ES 2.4.6 running at AWS in Frankfurt

We have a smaller instance of 8.13.2 running at AWS in Frankfurt. I think it's a 15 GB size -- good enough for testing. We can scale up the 8.x version as soon as we need to. Trying to keep costs down. It looks like the 2.4 is going to bill out at $3.80/hr They'll obviously cover all of the costs.6:55 PM Right now we are operating under a gentleman's agreement, but they'll see if it's possible to go beyond a handshake. ranguard: serverless will be available in v8. It's won't be GA for a while, but it sounds like we could get a preview of it. It looks like we'll need to create a snapshot of our current ES and then do a restore to the cloud. That would be the fast way. We have Kibana in both clusters This makes Elastic our biggest sponsor by quite a large margin, as far as monthly costs go.

Migrate to Hosted Elasticsearch

ranguard commented 4 months ago

As noted in https://github.com/metacpan/metacpan-api/discussions/1182

Using snapshots (which we already take) to AWS is probably the easiest path

https://github.com/metacpan/metacpan-api/blob/master/lib/MetaCPAN/Script/Snapshot.pm is the actual code