search

Icinga / icingaweb2-module-director

The Director aims to be your new favourite Icinga config deployment tool. Director is designed for those who want to automate their configuration deployment and those who want to grant their “point & click” users easy access to the configuration.
https://icinga.com/docs/director/latest
GNU General Public License v2.0
413 stars 203 forks source link

Automatic deployment job should recognize contemporarely manual deployment #2251

Closed mwaldmueller closed 2 years ago

mwaldmueller commented 3 years ago

Expected Behavior

Automatic deployment job should recognize contemporarely manual deployment.

Current Behavior

Automatic deployment job with contemporarely manual deployment leads to an error.

Possible Solution

Automatic deployment job should recognize contemporarely manual deployment and be postponed.

Steps to Reproduce (for bugs)

Your Environment

Thomas-Gelf commented 2 years ago

There is no good way to guarantee this without Icinga offering such functionality. An Icinga API method "Are you deploying?" also wouldn't help, as there might always be a race condition between the question and the trigger. So this would only work if Icinga would refuse the "deploy request" with a corresponding error message. However, config stages in Icinga have been built to deal with concurrency. So, this SHOULD not be required. If something goes wrong with them, this should be investigated and addressed.

A "lock" in the Director would of course reduce such woes to a minimum, but locks are dangerous. Should the Director forbid any deployment, unless Icinga returns a valid result for the former one? We're now dealing with lost deployments in a very robust way, but still - this could result in "endless" locks. Asks for timeouts. But then, for how long should we wait? This is tricky, and not worth the effort - as we can give no guarantees. Please also see #2412 for a related discussion.

We're moving more and more logic to the Director daemon, so as a side-effect of those efforts we'll probably be able to cover most such scenarios by pipe-lining deployments. You'll get the feature you're asking for, for free. There are just some other dependencies that need to be addressed first.

Still, this is nothing the Director can take responsibility for. It takes a couple of seconds to ship a large configurations, but it might take minutes unless the Icinga deployment is done. How long should we wait? Unless the daemon is once again up and running? Seems to still hurt sometimes. Unless IDO is ready? IcingaDB? We cannot cover all variants in a reliable way. Wait unless the state / startup log for the stage is ready? Sometimes a deployment gets lost in Icinga. And then? Wait forever?

Hope this helps, closing this issue for now.