[Feature]: Please add an official Restore-AzStorageBlob cmdlet for soft-deleted blobs

[milope]

Description of the new feature

Users are seemingly unable to have a simple Az PowerShell cmdlet to restore soft-deleted blobs on flat hierarchical namespace Storage Accounts. While we can run the following to achieve such result:

Get-AzStorageBlob -Context $storageAccountContext -Container $containerName -IncludeDeleted `
| Where-Object { $_.IsDeleted } `
| ForEach-Object {
    $_.BlobClient.Undelete()
}

This implementation doesn't take advantage of the LimitedConcurrencyTaskScheduler implementation used by other Az PowerShell Storage cmdlets like Remove-AzStorageBlob and would be ultimately much slower when recovering soft-deleted blobs in the order of millions.

Proposed implementation details (optional)

The cmdlet can be used to simplify soft-deleted blobs by making calls such as:

Get-AzStorageBlob -Context $storageAccountContext -Container $containerName -IncludeDeleted `
  | Where-Object { $_.IsDeleted } `
  | Restore-AzStorageBlob

[isra-fel]

Looping in Storage team to consider the feature request.

[microsoft-github-policy-service[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

[microsoft-github-policy-service[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

[yifanz7]

@milope For the performance issue, one possible solution is to leverage the ForEach-Object Parallel feature to speed up the restore operations. You can specify the throttle limit as mentioned in the doc.

For the script provided, adding the Parallel and ThrottleLimit parameters might look like:

Get-AzStorageBlob -Context  $storageAccountContext -Container $containerName -IncludeDeleted `
| Where-Object { $_.IsDeleted } `
| ForEach-Object -Parallel {
    $_.BlobClient.Undelete()
} -ThrottleLimit 10

Some tests we've done with restoring 1000 blobs of size 512 bytes:

• Without parallel: total time 00:05:34.9583022 (5min 34seconds)
• With parallel and throttle limit of 10: 00:00:37.3369275 (37 seconds)
• With parallel and throttle limit of 15: 00:00:29.2844898 (29 seconds)

As the key concern of the issue is the performance issue, we will also consider adding the new cmdlet in the future planning, but this might not be the top priority. Please let us know if you have any other questions.

[milope]

Foreach-Object -Parallel is not available in Windows PowerShell. I’ve already implemented my own Task Throttling, like the PowerShell code to support parallelism both in Windows PowerShell and PowerShell 7 to achieve a 1.5 million per 20-minute recovery.

The ask is more to address the fact there isn’t a simple cmdlet to do so and requires direct SDK access to make the call, which for novice users may cause more harm.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: yifanz7 @.> Sent: Monday, December 25, 2023 2:20:05 AM To: Azure/azure-powershell @.> Cc: Michael Lopez @.>; Mention @.> Subject: Re: [Azure/azure-powershell] [Feature]: Please add an official Restore-AzStorageBlob cmdlet for soft-deleted blobs (Issue #23750)

@milopehttps://github.com/milope For the performance issue, one possible solution is to leverage the ForEach-Object Parallel featurehttps://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/ to speed up the restore operations. You can specify the throttle limit as mentioned in the doc.

Get-AzStorageBlob -Context $storageAccountContext -Container $containerName -IncludeDeleted | Where-Object { $_.IsDeleted } | ForEach-Object -Parallel { $_.BlobClient.Undelete() } -ThrottleLimit 10

Some tests we've done with restoring 1000 blobs of size 512 bytes:

• Without parallel: total time 00:05:34.9583022 (5min 34seconds)
• With parallel and throttle limit of 10: 00:00:37.3369275 (37 seconds)
• With parallel and throttle limit of 15: 00:29.2844898 (29 seconds)

As the key concern of the issue is the performance issue, we will also consider adding the new cmdlet in the future planning, but this might not be the top priority. Please let us know if you have any other questions.

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-powershell/issues/23750#issuecomment-1868847654, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGC2UU5G7OIX26ZIJXZKWSTYLEZLLAVCNFSM6AAAAABAV4JMHKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYHA2DONRVGQ. You are receiving this because you were mentioned.Message ID: @.***>

[yifanz7]

@milope Thanks again for the feedback and good to know that a workaround has been found.

It's actually very common for users to call .NET SDK directly from PSH cmdlets. For example, creating a blob snapshot or setting the access tier of a blob also requires direct SDK access. And for this API to restore a blob, the usage $_.BlobClient.Undelete() is not too complicated for users as no parameters are required.

But yes we will definitely bring up the new cmdlet in the future planning, but probably not with top priority till more customer requests are raised.