moby / libnetwork

networking for containers
Apache License 2.0
2.16k stars 881 forks source link

libnetwork/overlay:fix join sandbox deadlock #2674

Closed corhere closed 1 year ago

corhere commented 1 year ago

Operations performed on overlay network sandboxes are handled by dispatching operations send through a channel. This allows for asynchronous operations to be performed which, since they are not called from within another function, are able to operate in an idempotent manner with a known/measurable starting state from which an identical series of iterative actions can be performed.

However, it was possible in some cases for an operation dispatched from this channel to write a message back to the channel in the case of joining a network when a sufficient volume of sandboxes were operated on.

A goroutine which is simultaneously reading and writing to an unbuffered channel can deadlock if it sends a message to a channel then waits for it to be consumed and completed, since the only available goroutine is more or less "talking to itself". In order to break this deadlock, in the observed race, a goroutine is now created to send the message to the channel.

(cherry picked from commit moby/moby@feab0cca9f9bb852df3372c7458b0d6d7b64d515)

thaJeztah commented 1 year ago

Oh! All green now; merging