migtools / mig-controller

OpenShift Migration Controller
Apache License 2.0
22 stars 42 forks source link

Don't let users run multiple migrations against the same clusters at the same time #733

Open djwhatle opened 4 years ago

djwhatle commented 4 years ago

Is your feature request related to a problem? Please describe. Right now, users are able to start migrations against 2 different plans at the same time.

This leads to issues because:

  1. Velero can't process requests in parallel
  2. Different migrations running at the same time could lead to swapping in different MigStorage credentials during an ongoing Velero Backup or Restore which could cause failure
  3. mig-controller will do things like ResticRestart mid-migration that will interrupt other migrations using restic.

Describe the solution you'd like

djwhatle commented 4 years ago

@jortel and I discussed this. We would solve as follows:

  1. Extend MigMigration postpone logic to ensure that two MigMigrations can't run at the same time if the src or target cluster referenced already has a migration running on it

  2. Consult with @sseago and @dymurray to determine if it's actually safe to create BSLs/VSLs/Cloud Secret ahead of time, or if we should move this logic to the MigMigration controller as part of the migration flow. The concern here: user could have multiple MigStorages in conflict with eachother. Since Velero has limitations around how it can only accept one set of cloud-credentials per cloud provider at a time, we have to be sensitive here.