search

anhnongdan / bimax_ha_migration

all db backup, recovery and migration scenarios for bimax
0 stars 0 forks source link

Monitor Server behavior when cron_backup is running #9

Closed anhnongdan closed 6 years ago

anhnongdan commented 7 years ago

PW4 VTV response 404 when backup is running.

PW4 down at x:50, almost always.

[2017_08_05_10_50] pw4  7 respond
[2017_08_05_10_55] pw4  7 respond
[2017_08_05_11_00] pw4  7 respond
[2017_08_05_11_05] pw4  7 respond
[2017_08_05_11_10] pw4  7 respond
[2017_08_05_11_15] pw4  7 respond
[2017_08_05_15_50] pw4  7 respond
[2017_08_05_16_50] pw4  7 respond
[2017_08_05_18_50] pw4  7 respond
[2017_08_05_19_50] pw4  7 respond
[2017_08_05_22_50] pw4  7 respond
[2017_08_05_23_50] pw4  7 respond
[2017_08_06_00_50] pw4  7 respond
[2017_08_06_01_50] pw4  7 respond
[2017_08_06_03_50] pw4  7 respond
[2017_08_06_05_50] pw4  7 respond
[2017_08_06_06_50] pw4  7 respond
[2017_08_06_08_50] pw4  7 respond
[2017_08_06_11_50] pw4  7 respond
[2017_08_06_13_50] pw4  7 respond
[2017_08_06_14_50] pw4  7 respond
[2017_08_06_15_50] pw4  7 respond
[2017_08_06_16_50] pw4  7 respond
[2017_08_06_17_50] pw4  7 respond
[2017_08_06_18_35] pw4  7 respond
[2017_08_06_18_40] pw4  7 respond
[2017_08_06_18_45] pw4  7 respond
[2017_08_07_00_50] pw4  7 respond
[2017_08_07_01_50] pw4  7 respond
[2017_08_07_03_50] pw4  7 respond
[2017_08_07_05_50] pw4  7 respond
[2017_08_07_06_50] pw4  7 respond
[2017_08_07_08_50] pw4  7 respond

Net interface look weird: screenshot from 2017-08-07 09-17-06

Rotation seems to work fine too: screenshot from 2017-08-07 09-38-11

anhnongdan commented 7 years ago

I stoped pw4 from 29 -31/07 for some reasons? => pw4 php-fpm max child back to 5

screen shot 2017-08-09 at 10 27 36 pm

anhnongdan commented 7 years ago

PW4 is down, cron back up seems stuck.

root@VT-HNI-BIMAX-02:/$ ps -ef | grep archive
root      1384  1117  0 Aug19 ?        00:00:00 bash /data/app/bimax/flow.sh archiveAll
root      1397  1384  0 Aug19 ?        00:01:37 bash /data/app/bimax/flow.sh archiveAll
root     30182 30126  3 05:45 ?        00:00:41 /usr/bin/xtrabackup --tables-exclude=(_archive_(blob|numeric)_temp_|_log_link_visit_action_2) --innodb-log-file-size=50331648 --backup -H 127.0.0.1 -P 3004 -u admin -p x xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --datadir=/data/bimax/pw4/db/data --target-dir=/data/bimax/pw4/db/backup
root     31805 30385  0 06:04 pts/0    00:00:00 grep --color=auto archive
anhnongdan commented 7 years ago

And this graph looks weird

screen shot 2017-08-26 at 6 11 18 am