Update swarm/manager Flavor drain to be idempotent

[kaufers]

When a manager node is removed, the following occurs:

• The associated flavor Drain function is invoked, this involves a node demotion and then a swarm node leave or a swarm node remove
• The instance plugin's Destroy function is invoked

If an error occurs in the processing then the instance may not be removed from the group. Currently, depending on the error, all future group operations that invoke a Drain will always fail because the Drain will error out if the node is not still a manager.

The fix is to make the Drain idempotent so that it return nil if the node has already been removed from the swarm. If the node is still in the swarm as a worker then it will be removed from the swarm.

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@c4a5634). Click here to learn what that means. The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #851   +/-   ##
=========================================
  Coverage          ?   48.52%           
=========================================
  Files             ?       89           
  Lines             ?     8015           
  Branches          ?        0           
=========================================
  Hits              ?     3889           
  Misses            ?     3770           
  Partials          ?      356

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c4a5634...b0c54e4. Read the comment docs.

[GordonTheTurtle]

Please sign your commits following these rules: https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work The easiest way to do this is to amend the last commit:

$ git clone -b "mgr-drain-idempotent" git@github.com:kaufers/infrakit.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354248336
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.