search

github / gh-ost

GitHub's Online Schema-migration Tool for MySQL
MIT License
12.43k stars 1.26k forks source link

Lock on Master db Server table due to gh-ost thread #742

Open haroon101 opened 5 years ago

haroon101 commented 5 years ago

Hi Shlomi ,

We are using gh-ost version is v1.0.47.

We have faced the situation where we observed the lock on the master db table by gh-ost thread that was in sleep state and it pilled up the other inserts.

Below is the alter that we have executed on the slave.

gh-ost --max-load=Threads_running=30 --critical-load=Threads_running=1000 --chunk-size=100 --throttle-control-replicas="dbs44,dbs46" --max-lag-millis=3000 --user="gh_user" --password="$$$$$" --host=localhost --database="xyz" --table="sales" --initially-drop-socket-file --initially-drop-ghost-table --initially-drop-old-table --verbose --alter="modify amount bigint unsigned DEFAULT '0',add info text DEFAULT NULL" --switch-to-rbr --allow-master-master --cut-over=default --exact-rowcount --concurrent-rowcount --default-retries=120 --nice-ratio=10 --hooks-path=/tmp/ --panic-flag-file=/tmp/ghost.panic.flag --postpone-cut-over-flag-file=/tmp/ghostcutover.file --execute

shlomi-noach commented 5 years ago

Hi! I'm sorry, there's not enough information here for me to work with. What happened? When? Can you identify what transaction was locking? What was gh-ost doing at that time, as per --debug logs?

haroon101 commented 5 years ago

Thanks, Shlomi for the quick response. unfortunately we don't have much logs, but in ghc table its showing below information.

Gh-ost was throttled at that time. and the lock was on the parent table and the gh-ost thread was in sleep state on Master.

shlomi-noach commented 5 years ago

@haroon101 I confess this isn't something I can solve given this information.