blackpiglet
commented
9 months ago I put some consideration to the BIA scenario here. The BIA is different because the Velero collects all cared resources at the start of the backup. Still, the BIAs are executed during the backup process, and return more additional resources also included in the backup.
That means the Velero server cannot determine the whole scope of the backup, until the backup finishes. That makes supporting parallel backups not possible because the Velero server cannot detect the backups' overlap and potential conflicts.
There is also some discussion about making the BIA add a new method to return the additional resources during the backup resource collecting stage. I think it cannot resolve the issue.
- First, many additional resources the BIAs care about are created during the BIA running, so it's not possible to know the additional resources before that, and I think it's safe for the parallel backup scenario because they will not cause any resource overlap.
- Second, the Velero server should do nothing other than archive the additional resources' YAML into the metadata file. Even the additional resources already existed in the metadata before the BIAs returned it, which should not do any harm to the other parallel backups.
I think the real problem BIA caused is that the Velero server cannot know what the BIAs do. If the BIA freezes the filesystem of a pod that is not included in the backup, although IMO it shouldn't happen, it will impact parallel filesystem backups.
Unfortunately, as an external binary, it's not possible to regulate the plugins' behavior. IMO, we can only give a guideline of how the plugins should work to make the parallel backups work.
reasonerjt
github-actions[bot]
kaovilai
Describe the problem/challenge you have
It's better to know the backed-up k8s resource's dependency. If the Velero server knows it, it can detect invalid backups before running the backup process. This feature can help to resolve the scenario described in PR #7045.
This feature also has benefits for
Describe the solution you'd like
The Velero server can use a DAG(Directed Acyclic Graphs) as the data structure to store the backup resources.
The DAG's content should be:
Say this string represents a DAG, The resource backup sequence should ordered from left to right.
The DAG should be generated by existing rules:
During generating the rules, if the later rules violate the existing DAG resource hierarchy, fail the backup, and warn the user the rule is invalid.
When taking the backup, it should start from the root node, and go through the root node's children. After that, traverse the children's children. If backup gets a resource, but the resource's parents are not all backed up yet, the Velero server should put it on hold, and go on, then the Velero server should retry with the on-hold resources before traversing the next layer of resources.
Anything else you would like to add:
Environment:
velero version):kubectl version):/etc/os-release):Vote on this issue!
This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.