github / gh-ost

GitHub's Online Schema-migration Tool for MySQL
MIT License
12.43k stars 1.26k forks source link

Is it bad practice to run gh-ost on a replica? #554

Open wluo opened 6 years ago

wluo commented 6 years ago

Hi,

I was having this discussion with someone from our SRE team who believe gh-ost should not be running on one of the replicas in our mysql cluster. But after searching around I couldn't find any explicit guidance on that.

I thought running gh-ost on the replica that's also generating the RBR replication log that gh-ost is tracking would skip a network hop and get the replication data to the gh-ost even faster. Further more, this is a replica that's not taking production traffic normally (it's our fallback read-only replica).

Is there any additional requirement we should know about to decide whether to run gh-ost on a replica or not other than the common-sense load/resource sharing considerations?

Thanks for open-sourcing gh-ost. We've been liking the tool a lot! :-)

shlomi-noach commented 6 years ago

There is no idiomatic location for running the gh-ost binary. We run it on a group of "utility" boxes, just because our MySQL production servers can come and go and change roles and it's easier if we know the set of servers where gh-ost runs on (and we can more easily wire chatops to those servers).

That said, running gh-ost on the replica makes perfect sense, and you are correct in observing it will skip a network hop (this hop is in particular important because it streams the entire replication log).

I think the decision falls under each company's best practices realm.

wluo commented 6 years ago

Thank you! It's good to get a confirmation.