search

openvstorage / framework

The Framework is a set of components and tools which brings the user an interface (GUI / API) to setup, extend and manage an Open vStorage platform.
Other
27 stars 23 forks source link

Easy way to restart OVS services #719

Closed wimpers closed 8 years ago

wimpers commented 8 years ago

Currently there is no easy way to restart all OVS services on a node. A command line might be useful. We can later integrate this in the support portal if needed.

pploegaert commented 8 years ago

there is a quick hack ovs stop services, but you might have to run it twice ... to be improved ...

wimpers commented 8 years ago

This one is for @kvanhijf ;-)

khenderick commented 8 years ago

@wimpers, what features do you want? Only the framework services or truly all services on a given node (e.g. volumedriver, dtls, proxies, asds, ...). Restarting all ovs services on a given node might be risky for e.g. running volumes or backend availability.

Restarting only framework services is already possible by just restarting ovs-watcher-framework (or starting or stopping it for that matter).

wimpers commented 8 years ago

I've created this one as @kvanhijf had to write a script to stop (all?) services. The reason I created this bug FR is to tackle this kind of scenario's where people have to write their own script.

@kvanhijf can you list the services which you had to stop?

khenderick commented 8 years ago

He wrote a script to stop ovs-watcher-framework on all nodes. We can write such shortcut (e.g. ovs stop framework or whatever. The important question is whether you want to stop all services or only the framework services.

khenderick commented 8 years ago

After discussion with @wimpers:

For now, no alternatives for other services (arakoons, etcds, volumedrivers, ...) as they have more impact and should thus be executed manually and carefully.

wimpers commented 8 years ago

After discussion with @khenderick

Restarting (proxies, volumedrivers, arakoons, etcd, ...) is tricky so we will start with the basics

ovs stop framework = stop ovs-watcher-service
ovs stop framework <ip> = stop ovs-watcher-service on that node
ovs stop framework all = stop ovs-watcher-service op all nodes

Same goes for restart and start of course.

Full cli will be picked up as part of https://github.com/openvstorage/home/issues/28

khenderick commented 8 years ago

Fixed by #1004, packaged in openvstorage-2.7.4-rev.4092.7a6ff12

JeffreyDevloo commented 8 years ago

Steps

Testing the commands and verifying using the GUI:

Output

ovs stop framework: Checking service:

root@ovs-node1:~# ovs stop framework
Stopping...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting

Browsing to 10.100.199.151: Got redirected to a different node that is online.

Check status of the node via GUI: Got warning mark within 30s: selection_113 Got failure mark within 120s: selection_114

ovs start framework: Checking service:

root@ovs-node1:~# ovs start framework
Starting...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework start/running, process 3710
root@ovs-node1:~#

Browsing to 10.100.199.151: page accessible Check status of the node via GUI: Got the OK mark within 5s: selection_115

ovs stop framework 10.100.199.151: Checking service:

root@ovs-node1:~# ovs stop framework 10.100.199.151
Stopping on ovs-node1...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting
root@ovs-node1:~#

Browsing to 10.100.199.151: Got redirected to another node that is online. Checking status of the node via GUI: Got warning mark within 30s: selection_113 Got failure mark within 120s: selection_114

ovs start framework 10.100.199.151: Checking service:

root@ovs-node1:~# ovs start framework 10.100.199.151
Starting on ovs-node1...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework start/running, process 4851
root@ovs-node1:~#

Browsing to 10.100.199.151: Page accessible. Check status of the node via GUI: Got the OK mark within 5s: selection_115

ovs stop framework all: Checking service:

root@ovs-node1:~# ovs stop framework all
Stopping on ovs-node2...
Stopping on ovs-node3...
Stopping on ovs-node1...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting
root@ovs-node2:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting
root@ovs-node3:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting

Browsing to any IP: No redirection (all nodes are down). ovs start framework all: Checking service:

root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework stop/waiting
root@ovs-node1:~# ovs start framework all
Starting on ovs-node2...
Starting on ovs-node3...
Starting on ovs-node1...
Done
root@ovs-node1:~# service ovs-watcher-framework status
ovs-watcher-framework start/running, process 5527
root@ovs-node2:~# service ovs-watcher-framework status
ovs-watcher-framework start/running, process 26324
root@ovs-node3:~# service ovs-watcher-framework status
ovs-watcher-framework start/running, process 25791

Browsing to all IP: All pages are accessible. Check status of the node via GUI: All nodes are marked as ok. selection_116

Test result

Test passed.

Packages: