Support stopping the failed pipeline and keep the test bed

[blackpiglet]

Describe the problem/challenge you have

It's possible that the E2E test case could fail. The current logic continues on the next test case, and the failed test environment is destroyed because the next test case environment is set up. Although the current E2E code already tries to print much information about the failed test, it's still important to let the developers or the testers have a chance to investigate the failed test bed.

Describe the solution you'd like

Suggest adding a new flag called --bail-out to tell the E2E script to skip the following tests when there is a failed test. --bail-out's default value is false. The feature is turned on when it's set to true.

Anything else you would like to add:

Environment:

• Velero version (use velero version):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• :+1: for "The project would be better with this feature added"
• :-1: for "This feature will not enhance the project in a meaningful way"

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands.

[blackpiglet]

unstale

[kaovilai]

I like this!