mdn / infra

(Deprecated) MDN Web Docs Infrastructure scripts and configuration
Mozilla Public License 2.0
52 stars 32 forks source link

Ability to connect to the production MySQL in US-west #467

Closed peterbe closed 3 years ago

peterbe commented 3 years ago

To migration from wiki.developer.mozilla.org to github.com/mdn/content we need to run our importer against a fresh copy of the MySQL data. In development, we've been using the daily database dumps from https://mana.mozilla.org/wiki/pages/viewpage.action?spaceKey=MDN&title=How+to%3A+use+Prod+data+locally This works, and is relatively fast once you've parsed the dump into a mysql server on localhost. However, it's bound to be hours late on the critical day of the final migration.

To run the importer using the read-replica hosted in Europe, on my home broadband here in US-east, it took 2h31min which is better than having to wait for the 24h daily backup.

We'd like to explore the possibility to connect directly from Peter's (US east) or Ryan's (US West) laptops to the production MySQL instance that runs in US-west. The hope is that this connection is faster and takes significantly less than 2h30min.

Our estimated day of the big migration is December 14, 2020. But it would be nice to try this sometime before in November or even now in October.

peterbe commented 3 years ago

I believe the solution only really needs to work for @escattone and myself, if that makes things simpler. Also, once we've executed this and fully switched over to Yari, we won't need this anymore.

limed commented 3 years ago

Done in https://github.com/mdn/infra/pull/469