`az extension list-available` is super slow - Store index.json somewhere else?

[o-l-a-v]

Describe the bug

az extension list-available is super slow, it takes seconds just to fetch index.json from https://aka.ms/azure-cli-extension-index-v1. Most likely because it's stored on a single region Azure storage account, blob storage. Related to:

• https://github.com/Azure/azure-cli/issues/19447

Here are some ideas to make it faster:

1. Store it somewhere else.
   • What about just pointing to index.json in the main branch?
   • Azure static web app with CDN?
2. Reduce file size of index.json: We're usually only interested in knowing what's the latest version available. Create a index-latest.json with just the latest stable and preview version? index.json will just continue growing.
3. Cache index.json locally, don't attempt to refetch it every CLI command that needs it.
   • Invalidate local version of index.json after 15 minutes?
   • Version index.json, then Azure CLI can check local cached version vs. what's available. Which should be much faster than fetching index.json. Use GitHub releases?

Related command

# Fastest - Use GitHub repo
$AvailableExtensions = [PSCustomObject](
    (Invoke-RestMethod -Method 'Get' -Uri 'https://raw.githubusercontent.com/Azure/azure-cli-extensions/main/src/index.json').'extensions'
)

# Use same URI as CLI
## Before redirect
$AvailableExtensions = [PSCustomObject](
    (Invoke-RestMethod -Method 'Get' -Uri 'https://aka.ms/azure-cli-extension-index-v1').'extensions'
)

## What redirect goes to
$AvailableExtensions = [PSCustomObject](
    (Invoke-RestMethod -Method 'Get' -Uri 'https://azcliextensionsync.blob.core.windows.net/index1/index.json').'extensions'
)

# Use CLI
$AvailableExtensions = [array](
    az extension list-available | ConvertFrom-Json
)

Errors

None

Issue script & Debug output

None

Expected behavior

Would be nice it if was faster

Environment Summary

Azure CLI v2.43.0

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[o-l-a-v]

Here's a quick test for getting latest version of bastion using Azure CLI v2.63.0, vs fetching index.json from this GitHub repo.

Click to view

```pwsh # Measure - Get latest version of "bastion" ## CLI - 6-7s Measure-Command -Expression { Write-Information -InformationAction 'Continue' -MessageData ( (az extension list-available | ConvertFrom-Json).Where{$_.'name' -eq 'bastion'}.'version' ) } | Select-Object -ExpandProperty 'TotalSeconds' ## Same URL as CLI - 5-6s Measure-Command -Expression { Write-Information -InformationAction 'Continue' -MessageData ( ( Invoke-RestMethod -Method 'Get' -Uri 'https://aka.ms/azure-cli-extension-index-v1' ` -Headers @{'content-encoding' = 'gzip'} -SslProtocol 'Tls13' ).'extensions'.'bastion'[-1].'metadata'.'version' ) } | Select-Object -ExpandProperty 'TotalSeconds' ## GitHub repo - 0.25-0.40s Measure-Command -Expression { Write-Information -InformationAction 'Continue' -MessageData ( ( Invoke-RestMethod -Method 'Get' -Uri 'https://raw.githubusercontent.com/Azure/azure-cli-extensions/main/src/index.json' ` -Headers @{'content-encoding' = 'gzip'} -SslProtocol 'Tls13' ).'extensions'.'bastion'[-1].'metadata'.'version' ) } | Select-Object -ExpandProperty 'TotalSeconds' ```

CLI takes about 6-7 seconds, while getting index.json from this GitHub repo then parse out the relevant information takes about 0.25-0.40 seconds.

Making this quicker can speed up all commands that needs to fetch index.json to find out what version is the latest, what version to install etc.