Open ydirson opened 1 month ago
Unfortunately XAPI doesn't have a transactional database, or support for transactions, so these race conditions are always possible. In this particular case 'allowed operations' -style locking could be used, to forbid further changes to the VM while the revert is running.
For some reason one of guests now has 2 CD VBDs, with the same
userdevice
. I understand that this is not supposed to happen, withxe
normally getting aDEVICE_ALREADY_EXISTS
error from XAPI:Despite this I now have VBDs on a guest violating this constraint:
xensource.log
shows for this VBD creation:This matches the XO code that (when requested to insert a CD, and after determining a guest does not have a CD VBD yet) queries XAPI for allowed VBD devices and creates one. Which seems to imply that on this months-old VM on which I used that CD VBD tens of times, this particular time XAPI hallucinated the lack of the CD VBD for long enough to let the XO XAPI client create a new, conflicting one and insert a VDI in it.
The symptom to the user then, since is that this 2nd CD drive, which had a VDI inserted at creation time, causes that VDI to be automatically inserted in the 1st CD drive every time the VM starts. I guess that one would fall into "undefined behavior" because we're in a state that's not supposed to be possible?
The log (and existing snapshot timestamps) shows this incident closely follows a VM.revert:
A few additional tests through XO show that, when using "VM.reset with the 'snapshot before' option activated", there is enough time to request a CD insertion before the notification of
VM.revert
ending comes in, and this time the extra VBD gets a distinctuserdevice
:I assume once the race condition is triggered, YMMV.
This is on XAPI 1.249.32 on XCP-ng 8.2.1.