Implemented the -m flag for parallel deletion of files in GitHub Actions.
Description
This PR introduces the use of the -m flag in the gsutil rm command within our GitHub Action. The addition of this flag is intended to speed up the deletion of multiple files, especially when there's a significant number of files to be removed from a GCS bucket. This enhancement is adaptive, ensuring efficient deletions irrespective of the number of files (many, one, or none).
Related Issue(s)
69
Motivation and Context
During our recent GitHub Action runs, we noticed that deleting files one-by-one can be time-consuming when dealing with multiple files. The Action itself suggested using the -m flag for efficiency. By integrating this flag, we aim to reduce the overall runtime of the GitHub Action, ensuring that file deletions are rapid and scalable.
Types of changes
What types of changes does your code introduce? Put an x in all the boxes that apply:
[x] New feature (non-breaking change which adds functionality)
Checklist:
Go over all the following points, and put an x in all the boxes that apply. If you're unsure about any of these, don't hesitate to ask. We're here to help!
[x] My code follows the code style of this project.
Summary
Implemented the
-mflag for parallel deletion of files in GitHub Actions.Description
This PR introduces the use of the
-mflag in thegsutil rmcommand within our GitHub Action. The addition of this flag is intended to speed up the deletion of multiple files, especially when there's a significant number of files to be removed from a GCS bucket. This enhancement is adaptive, ensuring efficient deletions irrespective of the number of files (many, one, or none).Related Issue(s)
69
Motivation and Context
During our recent GitHub Action runs, we noticed that deleting files one-by-one can be time-consuming when dealing with multiple files. The Action itself suggested using the
-mflag for efficiency. By integrating this flag, we aim to reduce the overall runtime of the GitHub Action, ensuring that file deletions are rapid and scalable.Types of changes
What types of changes does your code introduce? Put an
xin all the boxes that apply:Checklist:
Go over all the following points, and put an
xin all the boxes that apply. If you're unsure about any of these, don't hesitate to ask. We're here to help!