search

redhat-performance / quads

:calendar: The infrastructure deployment time machine
https://quads.dev
GNU General Public License v3.0
88 stars 36 forks source link

[BUG] Cloud is not set to provisioned=True if schedule is removed while move is in progress #361

Open grafuls opened 3 years ago

grafuls commented 3 years ago

Describe the bug There is a chance we might want to remove a schedule for a host moving into a new cloud while the move is still in progress. E.g.: All hosts on that cloud have already moved but 1 host is failing on the move step due to lack of network connectivity or broken state. If the schedule is removed, Quads has no chance to mark the cloud as provisioned=True due to the fact that the M&R action was not completed successfully. The consequence of this is that the cloud will never go into validation

To Reproduce Steps to reproduce the behavior:

  1. disable all crons
  2. create new schedule for several hosts
  3. setup one of those hosts to fail on move and rebuild (SSH disabled)
  4. run M&R and have it move all host to the new cloud while failing on the one setup for fail
  5. once the failed host is the only one left to move, remove the schedule for that host
  6. run validate_env and notice no clouds are available for validation

Expected behavior Cloud should go into validation despite host schedule being removed.

sadsfae commented 2 months ago

Closing this, not sure we've seen this for a few years now and we've been in this scenario a few times already.

sadsfae commented 2 months ago

This can be worked around by updating the provisioned boolean for assignments in Postgres for the affected assignment.

quads=# update assignments set provisioned = true where id = 54;
UPDATE 1

Documented here: https://github.com/redhat-performance/quads?tab=readme-ov-file#validating-after-removing-hosts