Closed nateshrevankar closed 10 years ago
The function for Start needs to be: Start Watchdog alone. Watchdog checks for a file for existence (this file to be created if AMF should NOT be restarted). Start AMF if restart file is not available as demon process. o Save previous core, log, db files (better to move this to Watchdog before starting SAF) o Load configuration of TIPC module. o Specify library paths o For System controller, start SNMP daemon o For Simulation, start GMS configurations o Start AMF by having node name and address. o Load HPI if it not simulation
The following things are accomplished in the uploaded files:
The initial WIP version is pushed in github: https://github.com/nateshrevankar/SAFplus-Availability-Scalability-Platform/commit/3ce40e7a2a5bd606df6e1709d5081f969fa1c31c
The basic testing is performed and it working well. further testing is in-progress and may need fine tuning as the watchdog and SAF start are inter-related.
The start-up process is verified and uploaded in location: https://github.com/nateshrevankar/SAFplus-Availability-Scalability-Platform/commit/382b04064fca111a803747b579b12add017375f7
The supported functionality are: start : Start Watchdog stop : Kill Wathdog restart : Stop ASP. After 30 sec - watchdog will start SAF again. status : ASP status. help : help menu.
Test Case 1: Procedure:
Test Case 2: Procedure:
Test Case 3: Procedure:
Test Case 4: Procedure:
Test Case 5: Procedure:
Result: PASS. root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 4212 6499 0 17:15 pts/3 00:00:00 grep --color=auto -i saf root 31172 1 0 15:01 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 32047 1 0 15:04 ? 00:00:07 /root/eval/bin/safplus_amf -c 0 -l 1 -n SCNodeI0 root 32061 32047 12 15:04 ? 00:16:47 /root/eval/bin/safplus_logd root 32072 32047 0 15:04 ? 00:00:06 /root/eval/bin/safplus_gms clGmsConfig.xml root 32089 32047 0 15:04 ? 00:00:06 /root/eval/bin/safplus_event root 32098 32047 0 15:04 ? 00:00:06 /root/eval/bin/safplus_name root 32099 32047 0 15:04 ? 00:00:07 /root/eval/bin/safplus_ckpt root 32126 32047 0 15:04 ? 00:00:06 /root/eval/bin/safplus_msg
Test Case 6: Procedure:
Test Case 7: Procedure:
Test Case 8: Procedure:
Test Case 9: Procedure:
Test Case 10: Procedure:
Test Case 11: Procedure:
Test Case 12: Procedure:
Test Case 13: Procedure:
Test Case 14: Procedure:
root@ubuntu:~/eval# ./eval restart DEBUG checkTipc: True WARNING ASP is not running on node [1]. Cleaning up anyway... INFO Waiting for AMF to shutdown... DEBUG checkTipc: True INFO Unloading TIPC ... DEBUG disable bearer :tipc-config -bd=eth:eth0 ...
Test Case 15: Procedure:
root@ubuntu:~/eval# ./eval stop DEBUG checkTipc: True WARNING ASP is not running on node [1]. Cleaning up anyway... INFO Waiting for AMF to shutdown... DEBUG checkTipc: True INFO Unloading TIPC ... DEBUG disable bearer :tipc-config -bd=eth:eth0 ...
Test Case 16: Procedure:
Test Case 17: Procedure:
Test Case 18: Procedure:
Test Case 19: Procedure:
Test Case 20: Procedure:
Test Case 21: Procedure:
Test Case 22: Procedure:
Test Case 23: Procedure:
Test Case 24: Procedure:
Expected result: AMF process not available and display appropriate message Result: PASS. root@ubuntu:~/eval# ./eval restart DEBUG checkTipc: True WARNING ASP is not running on node [1]. Cleaning up anyway... INFO Waiting for AMF to shutdown... DEBUG checkTipc: True INFO Unloading TIPC ... DEBUG disable bearer :tipc-config -bd=eth:eth0 ...
Test Case 25: Procedure:
Expected result: AMF process not available message is displayed and watchdog to start AMF within 30 sec. Result: PASS.
root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7447 1 0 18:49 ? 00:00:00 /root/eval/bin/safplus_amf -c 0 -l 1 -n SCNodeI0 root 7464 7447 0 18:49 ? 00:00:00 /root/eval/bin/safplus_logd root 7473 7447 0 18:49 ? 00:00:00 /root/eval/bin/safplus_gms clGmsConfig.xml root 7493 7447 1 18:49 ? 00:00:00 /root/eval/bin/safplus_event root 7502 7447 0 18:49 ? 00:00:00 /root/eval/bin/safplus_name root 7503 7447 1 18:49 ? 00:00:00 /root/eval/bin/safplus_ckpt root 7530 7447 0 18:49 ? 00:00:00 /root/eval/bin/safplus_msg root 7567 6499 0 18:49 pts/3 00:00:00 grep --color=auto -i saf
root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7447 1 0 18:49 ? 00:00:00 /root/eval/bin/safplus_amf -c 0 -l 1 -n SCNodeI0 root 7464 7447 2 18:49 ? 00:00:00 /root/eval/bin/safplus_logd root 7530 7447 0 18:49 ? 00:00:00 /root/eval/bin/safplus_msg root 7608 7597 0 18:49 pts/4 00:00:00 python /root/eval/etc/safplus_watchdog_start.py restart root 7637 6499 0 18:49 pts/3 00:00:00 grep --color=auto -i saf root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7464 1 2 18:49 ? 00:00:00 /root/eval/bin/safplus_logd root 7530 1 0 18:49 ? 00:00:00 /root/eval/bin/safplus_msg root 7608 7597 0 18:49 pts/4 00:00:00 python /root/eval/etc/safplus_watchdog_start.py restart root 7640 6499 0 18:49 pts/3 00:00:00 grep --color=auto -i saf root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7608 7597 0 18:49 pts/4 00:00:00 python /root/eval/etc/safplus_watchdog_start.py restart root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7784 6499 0 18:50 pts/3 00:00:00 grep --color=auto -i saf
root@ubuntu:~/eval/var/run# ps -eaf | grep -i saf root 7360 1 0 18:49 ? 00:00:00 python /root/eval/etc/safplus_watchdog.py root 7863 1 0 18:50 ? 00:00:00 /root/eval/bin/safplus_amf -c 0 -l 1 -n SCNodeI0 root 7878 7863 12 18:50 ? 00:00:32 /root/eval/bin/safplus_logd root 7888 7863 0 18:50 ? 00:00:00 /root/eval/bin/safplus_gms clGmsConfig.xml root 7905 7863 0 18:50 ? 00:00:00 /root/eval/bin/safplus_event root 7914 7863 0 18:50 ? 00:00:00 /root/eval/bin/safplus_name root 7915 7863 0 18:50 ? 00:00:00 /root/eval/bin/safplus_ckpt root 7950 7863 0 18:50 ? 00:00:00 /root/eval/bin/safplus_msg root 8118 6499 0 18:54 pts/3 00:00:00 grep --color=auto -i saf
The changes merged to master branch. https://github.com/OpenClovis/SAFplus-Availability-Scalability-Platform/commit/6ef206e3e5dcb901ea24775c03627e94ee6de8c1
Suggested Changes:
updated the corrections mentioned:
The git link is: https://github.com/OpenClovis/SAFplus-Availability-Scalability-Platform/commit/a042e0acda5ef1c2c4cd8305b70db062bc1fda4f
Updating code review comments for changed code in starting Watchdog and no_start file
The code review is tracked in: https://docs.google.com/a/openclovis.com/spreadsheet/ccc?key=0AjGC0u9a-NgKdHlvZHFZZ1dyckdEbE5wd3lrN3ZtSUE&usp=drive_web#gid=0
Roughly consolidation of LOC: Added: 10, Deleted: 80, Modified: 5
IN-PROGRESS: Further improvement for start-up python script in starting AMF in asp.py file by removing run time database.
I have updated changes in start procedure for Watchdog and SAFplus_AMF.
The change details are:
The files for review are:
The changed files are available in my branch in location: https://github.com/nateshrevankar/SAFplus-Availability-Scalability-Platform/commit/628953bcf4513082acd358dfb2d3dc3877ba80a5
completed
Initial comments from Andrew: Startup: “safplus start” should start the AMF watchdog. This program will start its normal operation. In doing so, it detects that SAFplus is not running and starts it. This start procedure should be the exact same code as happens after failure detection. AMF watchdog: The purpose of the AMF watchdog is to be an extremely simple program that restarts SAFplus if it stops. It needs to have ZERO bugs because nothing monitors it. The simplest way to make a zero bug program is to make it simple. Currently it is extremely complex and has been known to quit by raising unhandled exceptions or simply quitting for and unknown reason. The AMF watchdog needs to be rewritten to become a < 50 line program that: