search

Azure / azure-cli-extensions

Public Repository for Extensions of Azure CLI.
https://docs.microsoft.com/en-us/cli/azure
MIT License
382 stars 1.22k forks source link

az datafactory trigger wait is failing #3819

Open NestorRV opened 3 years ago

NestorRV commented 3 years ago

Describe the bug

The az datafactory trigger start is failing as well with a "Bad request error".

Command Name az datafactory trigger wait Extension Name: datafactory. Version: 0.5.0.

Errors:

The command failed with an unexpected error. Here is the traceback:
'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 428, in handler
    provisioning_state = self._get_provisioning_state(instance)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 468, in _get_provisioning_state
    provisioning_state = additional_properties.get('provisioningState')
AttributeError: 'NoneType' object has no attribute 'get'

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Expected Behavior

Environment Summary

Linux-5.11.0-31-generic-x86_64-with-debian-bullseye-sid
Python 3.6.10
Installer: DEB

azure-cli 2.24.2 *

Extensions:
subscription 0.1.4
azure-devops 0.18.0
datafactory 0.5.0
costmanagement 0.1.1
logic 0.1.2

Additional Context

ghost commented 3 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @shawnxzq, @lmy269, @Jingshu923.

Issue Details
## Describe the bug The `az datafactory trigger start` is failing as well with a "Bad request error". **Command Name** `az datafactory trigger wait Extension Name: datafactory. Version: 0.5.0.` **Errors:** ``` The command failed with an unexpected error. Here is the traceback: 'NoneType' object has no attribute 'get' Traceback (most recent call last): File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke cmd_result = self.invocation.execute(args) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute raise ex File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job result = cmd_copy(params) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__ return self.handler(*args, **kwargs) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 428, in handler provisioning_state = self._get_provisioning_state(instance) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 468, in _get_provisioning_state provisioning_state = additional_properties.get('provisioningState') AttributeError: 'NoneType' object has no attribute 'get' ``` ## To Reproduce: Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information. - _Put any pre-requisite steps here..._ - `az datafactory trigger wait --factory-name {} --resource-group {} --name {} --created` ## Expected Behavior ## Environment Summary ``` Linux-5.11.0-31-generic-x86_64-with-debian-bullseye-sid Python 3.6.10 Installer: DEB azure-cli 2.24.2 * Extensions: subscription 0.1.4 azure-devops 0.18.0 datafactory 0.5.0 costmanagement 0.1.1 logic 0.1.2 ``` ## Additional Context
Author: NestorRV
Assignees: -
Labels: `customer-reported`, `Data Factory`, `Service Attention`
Milestone: -
yonzhan commented 3 years ago

route to service team

shawnxzq commented 3 years ago

@yonzhan From the error, this is not a service side error, I would expect CLI team share more details on how service team can fix it, for example, how to change swagger, or what service side behavior is wrong and etc, thanks!

NestorRV commented 3 years ago

Any update on this @evelyn-ys @yonzhan ? Thanks!

evelyn-ys commented 3 years ago

Hi @shawnxzq, Since Triggers_Start is a LongRunningOperation from swagger definition, Service need to contain provisioningState in response when getting Triggers_Get for checking status async.

@NestorRV Could you share the debug log with --debug to make problem more clear?

NestorRV commented 3 years ago

Here you can see the debug output @evelyn-ys . I have replaced some parts with <...> syntax, I hope it's not a problem!

cli.knack.cli: Command arguments: ['datafactory', 'trigger', 'wait', '--factory-name', '<df_name>', '--resource-group', '<rg_name>', '--name', '<trigger_name>', '--created', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f615233abf8>, <function OutputProducer.on_global_arguments at 0x7f615226e730>, <function CLIQuery.on_global_arguments at 0x7f615228c840>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'datafactory': ['azext_datafactory']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: datafactory               0.005        15        74  /home/user/.azure/cliextensions/datafactory
cli.azure.cli.core: Total (1)                 0.005        15        74  
cli.azure.cli.core: Loaded 15 groups, 74 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : datafactory trigger wait
cli.azure.cli.core: Command table: datafactory trigger wait
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f6151d050d0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/user/.azure/commands/2021-09-09.08-28-39.datafactory_trigger_wait.380135.log'.
az_command_data_logger: command args: datafactory trigger wait --factory-name {} --resource-group {} --name {} --created --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f6151d2d158>, <function register_global_query_examples_argument.<locals>.register_query_examples at 0x7f6151c7b1e0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f6151c7b268>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f6151c7b378>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f615226e7b8>, <function CLIQuery.handle_query_parameter at 0x7f615228c8c8>, <function register_global_query_examples_argument.<locals>.handle_example_parameter at 0x7f6151c7b158>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f6151c7b2f0>]
az_command_data_logger: extension name: datafactory
az_command_data_logger: extension version: 0.5.0
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=DataFactoryManagementClient
cli.azure.cli.core.adal_authentication: AdalAuthentication.get_token invoked by Track 2 SDK with scopes=('https://management.core.windows.net//.default',)
cli.azure.cli.core._profile: Retrieving token from ADAL for resource 'https://management.core.windows.net/'
cli.azure.cli.core.util: attempting to read file /home/user/.azure/accessTokens.json as utf-8-sig
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - Authority:Performing instance discovery: ...
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - Authority:Performing static instance discovery
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - Authority:Authority validated via static instance discovery
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - TokenRequest:Getting token from cache with refresh if necessary.
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - CacheDriver:finding with query keys: {'_clientId': '...', 'userId': '...'}
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - CacheDriver:Looking for potential cache entries: {'_clientId': '...', 'userId': '...'}
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - CacheDriver:Found 10 potential entries.
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - CacheDriver:Resource specific token found.
adal-python: 2c9fca16-b326-49c8-8f27-f4ddfa1f9643 - CacheDriver:Returning token from cache lookup, AccessTokenId: b'...', RefreshTokenId: b'...'
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.DataFactory/factories/<df_name>/triggers/<trigger_name>?api-version=2018-06-01'
cli.azure.cli.core.sdk.policies: Request method: 'GET'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': '2b9e7f4f-1137-11ec-9b00-7b85b00edc02'
cli.azure.cli.core.sdk.policies:     'CommandName': 'datafactory trigger wait'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '--factory-name --resource-group --name --created --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.24.2 (DEB) azsdk-python-mgmt-datafactory/1.0.0 Python/3.6.10 (Linux-5.11.0-31-generic-x86_64-with-debian-bullseye-sid)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "GET /subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.DataFactory/factories/<df_name>/triggers/<trigger_name>?api-version=2018-06-01 HTTP/1.1" 200 None
cli.azure.cli.core.sdk.policies: Response status: 200
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Transfer-Encoding': 'chunked'
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies:     'Content-Encoding': 'gzip'
cli.azure.cli.core.sdk.policies:     'Expires': '-1'
cli.azure.cli.core.sdk.policies:     'Vary': 'Accept-Encoding'
cli.azure.cli.core.sdk.policies:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies:     'Server': 'Microsoft-IIS/10.0'
cli.azure.cli.core.sdk.policies:     'X-Powered-By': 'ASP.NET'
cli.azure.cli.core.sdk.policies:     'x-ms-ratelimit-remaining-subscription-reads': '11999'
cli.azure.cli.core.sdk.policies:     'x-ms-request-id': '6dc2129c-9b67-4dac-9760-739422a66593'
cli.azure.cli.core.sdk.policies:     'x-ms-correlation-request-id': '6dc2129c-9b67-4dac-9760-739422a66593'
cli.azure.cli.core.sdk.policies:     'x-ms-routing-request-id': 'FRANCESOUTH:20210909T062841Z:6dc2129c-9b67-4dac-9760-739422a66593'
cli.azure.cli.core.sdk.policies:     'Date': 'Thu, 09 Sep 2021 06:28:40 GMT'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"id":"/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.DataFactory/factories/<df_name>/triggers/<trigger_name>","name":"<trigger_name>","type":"Microsoft.DataFactory/factories/triggers","properties":{"annotations":[],"pipelines":[{"pipelineReference":{"referenceName":"<trigger_name>","type":"PipelineReference"},"parameters":{"Modelo":"Smarkia","triggerTime":"@trigger().scheduledTime"}}],"type":"ScheduleTrigger","typeProperties":{"recurrence":{"frequency":"Month","interval":1,"startTime":"2021-05-26T03:45:00Z","timeZone":"UTC","schedule":{}}},"runtimeState":"Started"},"etag":"34009a8e-0000-0d00-0000-6133b4a60000"}
cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
cli.azure.cli.core.util: Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 428, in handler
    provisioning_state = self._get_provisioning_state(instance)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 468, in _get_provisioning_state
    provisioning_state = additional_properties.get('provisioningState')
AttributeError: 'NoneType' object has no attribute 'get'

cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
cli.azure.cli.core.azclierror: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 428, in handler
    provisioning_state = self._get_provisioning_state(instance)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 468, in _get_provisioning_state
    provisioning_state = additional_properties.get('provisioningState')
AttributeError: 'NoneType' object has no attribute 'get'
az_command_data_logger: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 428, in handler
    provisioning_state = self._get_provisioning_state(instance)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 468, in _get_provisioning_state
    provisioning_state = additional_properties.get('provisioningState')
AttributeError: 'NoneType' object has no attribute 'get'
To open an issue, please run: 'az feedback'
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f6151d052f0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 1.070 seconds (init: 0.084, invoke: 0.986)
telemetry.save: Save telemetry record of length 6555 in cache
telemetry.check: Negative: The /home/user/.azure/telemetry.txt was modified at 2021-09-09 08:27:44.093864, which in less than 600.000000 s
evelyn-ys commented 3 years ago

Thanks @NestorRV!

@shawnxzq As you can see from the debug log, when we try to get provisioningState from Triggers_Get for the result of Triggers_Start, no state returned which caused CLI break:

cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.DataFactory/factories/<df_name>/triggers/<trigger_name>?api-version=2018-06-01'
cli.azure.cli.core.sdk.policies: Request method: 'GET'

cli.azure.cli.core.sdk.policies: Response status: 200
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"id":"/subscriptions/<subscription_id>/resourceGroups/<rg_name>/providers/Microsoft.DataFactory/factories/<df_name>/triggers/<trigger_name>","name":"<trigger_name>","type":"Microsoft.DataFactory/factories/triggers","properties":{"annotations":[],"pipelines":[{"pipelineReference":{"referenceName":"<trigger_name>","type":"PipelineReference"},"parameters":{"Modelo":"Smarkia","triggerTime":"@trigger().scheduledTime"}}],"type":"ScheduleTrigger","typeProperties":{"recurrence":{"frequency":"Month","interval":1,"startTime":"2021-05-26T03:45:00Z","timeZone":"UTC","schedule":{}}},"runtimeState":"Started"},"etag":"34009a8e-0000-0d00-0000-6133b4a60000"}
shawnxzq commented 3 years ago

@evelyn-ys This is not a resource create API, having a "provisioningState" is not meaningful, instead it's an operation against a resource. Please refer to the corresponding PS command for it, can I expect CLI team can fix it and make it work in the same way as PS? Thanks! https://docs.microsoft.com/en-us/powershell/module/az.datafactory/start-azdatafactoryv2trigger?view=azps-6.4.0

evelyn-ys commented 3 years ago

Since Triggers_Start is claimed to be a long-running-operation, CLI will allow command return immediately and let customer check the status later. Is there any state(seems not "provisioningState" in this case) we can refer to check the start operation result? @shawnxzq Or do you prefer az datafactory trigger start command not to support --no-wait any more?