Closed Nautigsam closed 1 month ago
Thank you for opening this issue, we will look into it.
Any updates on this? Our customers are hitting this as well. We have multiple reports of this. FYI @hprabh @anantshankar17
Hi, sorry for the inconvenience. You're right @Nautigsam, there is an incompatibility with the docker SDK and the requests package that ships with az cli version 2.62.0. A PR to fix this was merged with #29391 and will be in the next release. For the time being, downgrading the az cli to 2.61.0 will work
Hello @SethHollandsworth , after downgrading to 2.61.0
I seem to still have the issue. Also the . dockercfg
file seems not have been found:
+ az confcom acipolicygen --debug --approve-wildcards -a /home/runner/work/uid2-operator/uid2-operator/deployment-artifacts/operator.json --print-policy
DEBUG: cli.knack.cli: Command arguments: ['confcom', 'acipolicygen', '--debug', '--approve-wildcards', '-a', '/home/runner/work/uid2-operator/uid2-operator/deployment-artifacts/operator.json', '--print-policy']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fc73fc3f100>, <function OutputProducer.on_global_arguments at 0x7fc73fb32840>, <function CLIQuery.on_global_arguments at 0x7fc73f8d0360>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Command index version or cloud profile is invalid or doesn't match the current command.
DEBUG: cli.azure.cli.core: Command index has been invalidated.
DEBUG: cli.azure.cli.core: No module found from index for '['confcom', 'acipolicygen', '--debug', '--approve-wildcards', '-a', '/home/runner/work/uid2-operator/uid2-operator/deployment-artifacts/operator.json', '--print-policy']'
DEBUG: cli.azure.cli.core: Loading all modules and extensions
DEBUG: cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'compute_recommender', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands
DEBUG: cli.azure.cli.core: acr 0.106 36 149
DEBUG: cli.azure.cli.core: acs 0.014 14 76
DEBUG: cli.azure.cli.core: advisor 0.001 3 6
DEBUG: cli.azure.cli.core: ams 0.004 22 100
DEBUG: cli.azure.cli.core: apim 0.004 14 69
DEBUG: cli.azure.cli.core: appconfig 0.002 9 47
DEBUG: cli.azure.cli.core: appservice 0.037 79 270
DEBUG: cli.azure.cli.core: aro 0.007 1 10
DEBUG: cli.azure.cli.core: backup 0.003 16 60
DEBUG: cli.azure.cli.core: batch 0.019 34 102
DEBUG: cli.azure.cli.core: batchai 0.002 10 30
DEBUG: cli.azure.cli.core: billing 0.007 19 52
DEBUG: cli.azure.cli.core: botservice 0.002 12 42
DEBUG: cli.azure.cli.core: cdn 0.082 8 49
DEBUG: cli.azure.cli.core: cloud 0.001 1 7
DEBUG: cli.azure.cli.core: cognitiveservices 0.002 10 33
DEBUG: cli.azure.cli.core: compute_recommender 0.002 1 1
DEBUG: cli.azure.cli.core: config 0.001 2 7
DEBUG: cli.azure.cli.core: configure 0.002 2 5
DEBUG: cli.azure.cli.core: consumption 0.010 8 9
DEBUG: cli.azure.cli.core: container 0.006 1 11
DEBUG: cli.azure.cli.core: containerapp 0.103 36 115
DEBUG: cli.azure.cli.core: cosmosdb 0.009 58 199
DEBUG: cli.azure.cli.core: databoxedge 0.005 5 28
DEBUG: cli.azure.cli.core: dla 0.002 23 62
DEBUG: cli.azure.cli.core: dls 0.002 7 41
DEBUG: cli.azure.cli.core: dms 0.001 3 22
DEBUG: cli.azure.cli.core: eventgrid 0.003 25 96
DEBUG: cli.azure.cli.core: eventhubs 0.007 13 19
DEBUG: cli.azure.cli.core: extension 0.001 1 7
DEBUG: cli.azure.cli.core: feedback 0.000 1 2
DEBUG: cli.azure.cli.core: find 0.001 1 1
DEBUG: cli.azure.cli.core: hdinsight 0.005 8 39
DEBUG: cli.azure.cli.core: identity 0.001 2 11
DEBUG: cli.azure.cli.core: interactive 0.000 1 1
DEBUG: cli.azure.cli.core: iot 0.055 19 82
DEBUG: cli.azure.cli.core: keyvault 0.006 20 113
DEBUG: cli.azure.cli.core: kusto 0.001 3 14
DEBUG: cli.azure.cli.core: lab 0.002 11 34
DEBUG: cli.azure.cli.core: managedservices 0.001 3 8
DEBUG: cli.azure.cli.core: maps 0.001 5 13
DEBUG: cli.azure.cli.core: marketplaceordering 0.002 1 2
DEBUG: cli.azure.cli.core: monitor 0.327 18 60
DEBUG: cli.azure.cli.core: mysql 0.069 15 51
DEBUG: cli.azure.cli.core: netappfiles 0.025 8 17
DEBUG: cli.azure.cli.core: network 0.151 103 337
DEBUG: cli.azure.cli.core: policyinsights 0.007 9 17
DEBUG: cli.azure.cli.core: privatedns 0.012 14 60
DEBUG: cli.azure.cli.core: profile 0.001 2 8
DEBUG: cli.azure.cli.core: rdbms 0.014 49 202
DEBUG: cli.azure.cli.core: redis 0.002 7 38
DEBUG: cli.azure.cli.core: relay 0.013 7 8
DEBUG: cli.azure.cli.core: resource 0.009 51 231
DEBUG: cli.azure.cli.core: role 0.002 17 61
DEBUG: cli.azure.cli.core: search 0.005 7 19
DEBUG: cli.azure.cli.core: security 0.006 47 98
DEBUG: cli.azure.cli.core: servicebus 0.005 12 15
DEBUG: cli.azure.cli.core: serviceconnector 0.027 20 307
DEBUG: cli.azure.cli.core: servicefabric 0.010 27 80
DEBUG: cli.azure.cli.core: signalr 0.002 9 34
DEBUG: cli.azure.cli.core: sql 0.010 56 215
DEBUG: cli.azure.cli.core: sqlvm 0.017 4 20
DEBUG: cli.azure.cli.core: storage 0.034 59 273
DEBUG: cli.azure.cli.core: synapse 0.009 54 246
DEBUG: cli.azure.cli.core: util 0.001 3 7
DEBUG: cli.azure.cli.core: vm 0.036 58 233
DEBUG: cli.azure.cli.core: Total (66) 1.317 1204 4681
DEBUG: cli.azure.cli.core: Loading extensions:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands Directory
DEBUG: cli.azure.cli.core: confcom 0.001 1 2 /opt/az/azcliextensions/confcom
DEBUG: cli.azure.cli.core: Total (1) 0.001 1 2
DEBUG: cli.azure.cli.core: Loaded 1191 groups, 4683 commands.
DEBUG: cli.azure.cli.core: Updated command index in 0.003 seconds.
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7fc73dee0ea0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-07-25.01-21-33.confcom_acipolicygen.49.log'.
INFO: az_command_data_logger: command args: confcom acipolicygen --debug --approve-wildcards -a {} --print-policy
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7fc73df26340>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7fc73df35580>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7fc73df356c0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7fc73b47[144](https://github.com/IABTechLab/uid2-operator/actions/runs/10086443167/job/27888999705#step:24:145)0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7fc73fb328e0>, <function CLIQuery.handle_query_parameter at 0x7fc73f8d0400>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7fc73df35620>]
INFO: az_command_data_logger: extension name: confcom
INFO: az_command_data_logger: extension version: 0.3.5
DEBUG: docker.utils.config: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
DEBUG: docker.utils.config: No config file found
DEBUG: docker.utils.config: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
DEBUG: docker.utils.config: No config file found
WARNING: cli.azext_confcom.custom: The docker process was not found. Please start Docker.
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fc73dee1120>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.__main__: Command ran in 1.805 seconds (init: 0.252, invoke: 1.553)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3654 in cache
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/usr/local/bin/python /usr/local/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /root/.azure"
INFO: telemetry.process: Return from creating process
INFO: telemetry.main: Finish creating telemetry upload process.
Do you have any insight what might have gone wrong?
Hey @cYKatherine! for .dockercfg
not being found, I've seen a couple different possibilities: the file doesn't exist because it's not needed (not authenticating to a registry, etc.) or the owner of the file is root and we don't have access
For the Docker process not found
issue, I'd double check that the docker service is started with systemctl | grep running
. If that looks fine, the involved way to debug would be:
start interactive python
python3
open a client and see if it errors out
import docker
client = docker.from_env()
client.images.list()
the version of docker-py that comes with confcom v0.3.5 is 7.1.0, not sure if that's important. If all that checks out, lmk and we can dig further
Hello @SethHollandsworth , thank you so much.
Hi, sorry for the inconvenience. You're right @Nautigsam, there is an incompatibility with the docker SDK and the requests package that ships with az cli version 2.62.0. A PR to fix this was merged with https://github.com/Azure/azure-cli/pull/29391 and will be in the next release. For the time being, downgrading the az cli to 2.61.0 will work
I ran az confcom acipolicygen --debug --approve-wildcards -a xxx.json --print-policy
in a github workflow. Originally it worked with the latest version in a simple run script, but because of the bug mentioned above, I have to downgrade to 2.61.0.
To downgrade it to 2.61.0, I have to use the azure/cli@v2
action to specify the 2.61.0 version. However, it seems that this action doesn't allow calling docker cli: https://github.com/Azure/cli/issues/152, https://github.com/Azure/cli/issues/156
I'm just wondering if my understanding is correct. If so do you have any insight how to bypass this issue? Or can I expect an estimate on the new release that fixes the confcom
extension bug?
Thank you so much!
Hi @cYKatherine, I didn't realize it was in a Github action. To get around needing docker, you could use the --tar
option in confcom
which takes in a path to a tar file to hash images locally. But then there would need to be a way to put the tar file into the action's filesystem. In my opinion, this would be more pain than it's worth.
The AZ CLI looks like it makes a release ~monthly, so it'll hopefully only be a week or two until the next version with the requests
package patched. Hope this helps, best of luck!
Thank you so much! @SethHollandsworth
@SethHollandsworth / @yonzhan : Do we have an update on the release date for 2.63?
Hi all @cYKatherine @shiswara @Nautigsam @anrdesai , v2.63.0 was released a few days ago and I confirmed it works with confcom
!
To upgrade please use: az upgrade
https://github.com/Azure/azure-cli/releases/tag/azure-cli-2.63.0
Describe the bug
When using confcom extension, I get this error:
This message hides the real error raised by
docker
package:It seems to be related to this issue. I suspect there is a conflict in the versions of
docker
orrequests
.Related command
az confcom acipolicygen --debug --template-file "***.json"
Errors
The docker process was not found. Please start Docker.
Issue script & Debug output
Expected behavior
The command should execute successfully as it did in version 2.61.0.
Environment Summary
azure-cli 2.62.0
core 2.62.0 telemetry 1.1.0
Extensions: confcom 0.3.5
Dependencies: msal 1.28.1 azure-mgmt-resource 23.1.1
Python location '/opt/azure-cli/bin/python' Extensions directory '/home/***/.azure/cliextensions'
Python (Linux) 3.12.4 (main, Jun 7 2024, 06:33:07) [GCC 14.1.1 20240522]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
Additional context
I use Arch and I installed the
azure-cli
package.