search

actions / runner-images

GitHub Actions runner images
MIT License
9.78k stars 3k forks source link

`az` behaves similarly on Ubuntu as it did on windows #8442

Closed jessehouwing closed 9 months ago

jessehouwing commented 11 months ago

Description

See:

Platforms affected

Runner images affected

Image version and build link

Image: ubuntu-22.04 Version: 20230924.1.0 Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230924.1/images/linux/Ubuntu2204-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230924.1

Is it regression?

No

Expected behavior

No rebuild of the command cache when running az command for the first time

Logs should read:

DEBUG: cli.knack.cli: Command arguments: ['devops', 'login', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f4e8dd393f0>, <function OutputProducer.on_global_arguments at 0x7f4e8dca0310>, <function CLIQuery.on_global_arguments at 0x7f4e8dac5510>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'devops': ['azext_devops']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: Total (0)                 0.000         0         0
DEBUG: cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
DEBUG: cli.azure.cli.core: Loading extensions:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
DEBUG: cli.azure.cli.core: azure-devops              0.077        60       192  /opt/az/azcliextensions/azure-devops
DEBUG: cli.azure.cli.core: Total (1)                 0.077        60       192  
DEBUG: cli.azure.cli.core: Loaded 60 groups, 192 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : devops login
DEBUG: cli.azure.cli.core: Command table: devops login

No installation of keychain when running az devops login for the first time

Logs should read:

INFO: az_command_data_logger: extension name: azure-devops
INFO: az_command_data_logger: extension version: 0.26.0
DEBUG: cli.knack.prompting: No tty available.
INFO: cli.azext_devops.dev.team.credentials: Getting PAT token in non-interactive mode.
INFO: keyring.backend: Loading KWallet
INFO: keyring.backend: Loading SecretService
INFO: keyring.backend: Loading Windows
INFO: keyring.backend: Loading chainer
INFO: keyring.backend: Loading macOS
DEBUG: cli.azext_devops.dev.common.credential_store: Keyring backend : keyring.backends.fail.Keyring (priority: 0)

Actual behavior

Calling az devops login results in a command index rebuild:

DEBUG: cli.knack.cli: Command arguments: ['devops', 'login', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fb941d113f0>, <function OutputProducer.on_global_arguments at 0x7fb941c78310>, <function CLIQuery.on_global_arguments at 0x7fb941a9d510>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Command index version or cloud profile is invalid or doesn't match the current command.
DEBUG: cli.azure.cli.core: Command index has been invalidated.
DEBUG: cli.azure.cli.core: No module found from index for '['devops', 'login', '--debug']'
DEBUG: cli.azure.cli.core: Loading all modules and extensions
DEBUG: cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'config', 'configure', 'consumption', 'container', 'cosmosdb', 'databoxedge', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
DEBUG: cli.azure.cli.core: Loading command modules:

And installation of keychain:

INFO: az_command_data_logger: extension name: azure-devops
INFO: az_command_data_logger: extension version: 0.26.0
DEBUG: cli.knack.prompting: No tty available.
INFO: cli.azext_devops.dev.team.credentials: Getting PAT token in non-interactive mode.
DEBUG: cli.azext_devops.dev.common.pip_helper: installing keyring~=17.1.1
DEBUG: cli.azure.cli.core.extension.operations: Running: ['/usr/bin/../../opt/az/bin/python3', '-m', 'pip', 'install', 'keyring~=17.1.1', '--target', '/opt/az/azcliextensions/azure-devops', '-vv', '--disable-pip-version-check', '--no-cache-dir']
DEBUG: cli.azure.cli.core.extension.operations: Using pip 23.2.1 from /opt/az/lib/python3.10/site-packages/pip (python 3.10)

Repro steps

In Azure Pipelines run the following workflow:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- pwsh: |
      echo "$env:ACCESSTOKEN" | az devops login --debug
      az pipelines runs show --id $env:BUILD_BUILDID --query "definition.id" --verbose --debug --organization $env:SYSTEM_COLLECTIONURI --project $env:SYSTEM_TEAMPROJECT
  env:
    ACCESSTOKEN: $(System.AccessToken)
ilia-shipitsin commented 11 months ago

it looks like moving complexity from Azure CLI to runner images, can this be addressed on Azure CLI (which I believe makes more sense)

jessehouwing commented 11 months ago

Not sure how actually, they optimize for a different goal than you.

So the bits that redirect the caching folder should be in the runner image.

The installation of keyring, I completely agree should be in az devops by default, but as long as they make it optional, the runner should trigger that option.

The Python warm-up thing, is something I feel they should just stick in the installer.

I've files issues with the az team, but it's been hard getting through to them.

For now, this is a 30 line fix. You could probably skip the az --help calls, since python is already much faster on ubuntu and make it a 15 line fix. Not sure it'll ever be much smaller.

ilia-shipitsin commented 11 months ago

initially I thought that we would keep warm up for Windows no longer than it will be implemented on Azure CLI side.

jessehouwing commented 11 months ago

Yeah, the calls to az --help and az devops --help should probably be the thing that can go if it is implemented in az. But for now, extension installation doesn't trigger any code in the extension. And that might be a security feature, I haven't checked, but not immediately executing code downloaded from the webs is probably a good thing.

With regards to the perf on Windows, it's a shame that Python is a lot slower on Windows than it is on linux. The Windows image clearly has a lot more to gain. Then again... 13s... It's another 13s.

image

mikhailkoliada commented 9 months ago

Closing as commented at https://github.com/actions/runner-images/pull/8441#issuecomment-1849075087

jessehouwing commented 9 months ago

@mikhailkoliada if I remove the folder redirection? I think this should be possible on Linux. The behavior on windows would differ from linux thiugh, since windows does set all these environment variables.

Would it make sense to keep the cleanup routine? Cause now running the tests leaves telemetry and command history data from provisioning.