ceph / ceph-cookbook

Chef cookbooks for Ceph
Apache License 2.0
100 stars 108 forks source link

Down OSDs are (inadvertantly) brought up by chef-client #229

Open mdsteveb opened 8 years ago

mdsteveb commented 8 years ago

This behavior has caused problems for me in a few ways.

We have chef-client running every 30 minutes (daemonized). Every time it runs, it brings up all OSDs, even ones that have been marked down due to starting to fail (resulting in flapping up/down), or manually downed for whatever reason. This is pretty anti-social. (This morning I had to take a couple nodes' worth of OSDs down for a network outage, but didn't want to take the entire systems down so that I could verify network had been restored properly when it was finished; only to find that they'd already come back up on their own which was definitely not what I wanted or intended.)

Are there any parameters I can tweak to prevent this? How are others dealing with this? I know I can probably kill chef-client on the node for the duration but this isn't exactly optimal either.