search

vatesfr / xen-orchestra

The global orchestration solution to manage and backup XCP-ng and XenServer.
https://xen-orchestra.com
Other
782 stars 264 forks source link

Mirror backup job issue #7955

Open rtjdamen opened 1 month ago

rtjdamen commented 1 month ago

Are you using XOA or XO from the sources?

XOA

Which release channel?

latest

Provide your commit number

No response

Describe the bug

Since latest update we are able to filter backups from repository that you do want to mirror. For some reason this is still reading all vms from the repository and not only the selected ones, it will then exclude them but this is time consuming. Also if one of these backups is currently processed by an recovery job is fails these vms with error "Lock file is already being held" also if this vm is excluded by the filter.

Error message

Lock file is already being held

To reproduce

  1. Create a mirror job and select only specific vms
  2. Run a normal job on this repository not contained the vms from step 1
  3. run the mirror job
  4. See error "Lock file is already being held"

Expected behavior

it should not try to process vms that are not part of this filter, it's also io intensive from what i can see if it is checking all vms.

Screenshots

No response

Node

18.20.2

Hypervisor

XCP-NG 8.2.1

Additional context

No response

julien-f commented 1 month ago

There is no easy answer on this, to help mitigating this issue you can disable the merge worker so that the merge step happens during the backup run itself.

This must be done manually in the config file:

https://github.com/vatesfr/xen-orchestra/blob/40f5f48f172f0b5ce83814c83d8a562ba7d3558d/packages/xo-server/config.toml#L81

rtjdamen commented 1 week ago

Hi Julien,

I understand this however if u create the mirror backups with a different instance this is not going to work. Also i believe the mirror job does not need to process the filtered vms, it currenlty tries everything also the ones filtered out, it does not process any data however. The job should not touch them at all. It now fails without a reason on a vm that is not even included.

olivierlambert commented 1 week ago

Are you meaning that you use another XO to make a mirror job? I don't think this was ever planned when designing the feature.

rtjdamen commented 1 week ago

Are you meaning that you use another XO to make a mirror job? I don't think this was ever planned when designing the feature.

I understand, but the behavior of this issue also occurs when u run it on the same xoa instance, so it’s not related.