search

Azure / azure-cli

Azure Command-Line Interface
MIT License
4.01k stars 2.98k forks source link

az dls fs access set-entry errors out in powershell, bash and shell #14116

Open tbojanowski opened 4 years ago

tbojanowski commented 4 years ago

Originally reported in DevOps, but they don't think its their problem.

https://developercommunity.visualstudio.com/content/problem/1090879/az-dls-fs-access-set-entry-errors-out-in-powershel.html

While running manually in Azure web Shell, command succeeds and returns no errors. While running the same command in Azure DevOps release pipeline with fail on standard error turned off: Powershell succeeds in adding access but returns error and fails entire deployment. Bash fails to add access, returns error and fails deployment Shell fails to add access, returns error but doesn't fail deployment All other CLI commands in the script run successfully, its just this one that fails and only in deployment pipeline. I'm running out of ideas, please help I can't move forward with this issue.

Powershell: az dls fs access set-entry --account datalakeName --acl-spec group:"test-datalake-store-dev":-w --path /

Bash: az dls fs access set-entry --account %1 --acl-spec group:"%2":-w --path /

Shell: az dls fs access set-entry --account $1 --acl-spec group:"$2":-w --path /

I was able to bypass this issue by using straight AZ Powershell, but Cli command keeps failing. $SecurityGroupId = (Get-AzADGroup -SearchString $DataLakeSecurityGroupName).Id Set-AzDataLakeStoreItemAclEntry -AccountName "$DataLakeName" -Path / -AceType Group -Id $SecurityGroupId -Permissions All

I did try using --only-show-errors at the end of the command, without any success in pipeline. In Azure portal I open the Azure Cloud Shell with Powershell mode and run this, then verify and user or group was added with access correctly.

PS /home/username> az dls fs access set-entry --account datalakestorename --acl-spec user:"username":-w- --path / Command group 'dls' is in preview. It may be changed/removed in a future release. PS /home/username>

cli in DevOps Error output :

2020-06-24T13:48:04.6936772Z D:\a\r1\a>REM Normal code goes here 2020-06-24T13:48:04.6937157Z 2020-06-24T13:48:04.6937859Z D:\a\r1\a>ECHO setting reporting datalakeName access policy for test-datalake-store-dev 2020-06-24T13:48:04.6938439Z setting reporting datalakeName access policy for test-datalake-store-dev 2020-06-24T13:48:04.6938792Z 2020-06-24T13:48:04.6941676Z D:\a\r1\a>az dls fs access set-entry --account datalakeName --acl-spec group:"test-datalake-store-dev":-w --path / 2020-06-24T13:48:06.1569780Z Command group 'dls' is in preview. It may be changed/removed in a future release. 2020-06-24T13:48:07.5705928Z The command failed with an unexpected error. Here is the traceback: 2020-06-24T13:48:07.5706266Z 2020-06-24T13:48:07.5724150Z Data-lake REST exception: MODIFYACLENTRIES, . 2020-06-24T13:48:07.5724548Z Traceback (most recent call last): 2020-06-24T13:48:07.5724939Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\knack\knack\cli.py", line 215, in invoke 2020-06-24T13:48:07.5725559Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands__init.py", line 654, in execute 2020-06-24T13:48:07.5726177Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands__init.py", line 718, in _run_jobs_serially 2020-06-24T13:48:07.5726758Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands__init.py", line 711, in _run_job 2020-06-24T13:48:07.5727281Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\six\six.py", line 703, in reraise 2020-06-24T13:48:07.5727795Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\init__.py", line 688, in _run_job 2020-06-24T13:48:07.5728369Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\commands\init.py", line 325, in call__ 2020-06-24T13:48:07.5728946Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\core\init__.py", line 545, in default_command_handler 2020-06-24T13:48:07.5729566Z File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure\cli\command_modules\dls\custom.py", line 304, in set_adls_item_acl_entry 2020-06-24T13:48:07.5730191Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\core.py", line 509, in modify_acl_entries 2020-06-24T13:48:07.5730816Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\core.py", line 459, in _acl_call 2020-06-24T13:48:07.5731409Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\lib.py", line 454, in call 2020-06-24T13:48:07.5732009Z File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-_kvuguic\azure-datalake-store\azure\datalake\store\lib.py", line 349, in log_response_and_raise 2020-06-24T13:48:07.5732599Z azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, . 2020-06-24T13:48:07.5732895Z 2020-06-24T13:48:07.5733107Z To open an issue, please run: 'az feedback' 2020-06-24T13:48:07.7208548Z ##[error]Script failed with exit code: 1 2020-06-24T13:48:07.7448489Z [command]C:\windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account clear" 2020-06-24T13:48:09.4233766Z ##[section]Finishing: data lake store access

yonzhan commented 4 years ago

add to S173

matendie commented 4 years ago

I found another issue, same problem, command works in web terminal but it fails in DevOps release pipeline I'm not sure if your coming up fix will repair this too or not, but figure you may want to know this is widespread issue.

command: $dlaExists = az dla account list --subscription $SubscriptionId --query "[?contains(name, '$DataLakeAnalyticsName')].[name]" --output tsv

error output: 2020-07-14T13:09:25.7274118Z [command]C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'd:\a_temp\azureclitaskscript1594732156968.ps1'" 2020-07-14T13:09:33.8864291Z az : Command group 'dla' is in preview. It may be changed/removed in a future release. 2020-07-14T13:09:33.8872287Z At D:\a\r1\a\foldername\projectname\foldername\DataLake.ps1:65 char:15 2020-07-14T13:09:33.8873344Z + ... laExists = az dla account list --subscription '12345678-1234-1234-12 ... 2020-07-14T13:09:33.8874158Z + ~~~~~~~~~~~~~ 2020-07-14T13:09:33.8875354Z + CategoryInfo : NotSpecified: (Command group '...future release.:String) [], RemoteException 2020-07-14T13:09:33.8876222Z + FullyQualifiedErrorId : NativeCommandError 2020-07-14T13:09:33.8876759Z
2020-07-14T13:09:33.9244178Z ##[error]Script failed with exit code: 1

I created new bug ticket here: https://github.com/Azure/azure-cli/issues/14359

ghost commented 4 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sumantmehtams.

Juliehzl commented 4 years ago

@akharit could you also take a look? I can reproduce with current azure cli.

The command failed with an unexpected error. Here is the traceback:

Data-lake REST exception: MODIFYACLENTRIES, .
Traceback (most recent call last):
  File "E:\Microsoft\AzureCLI\azure-cli\env\lib\site-packages\knack\cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 654, in execute
    raise ex
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "E:\Microsoft\AzureCLI\azure-cli\env\lib\site-packages\six.py", line 703, in reraise
    raise value
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\commands\__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli-core\azure\cli\core\__init__.py", line 776, in default_command_handler
    return op(**command_args)
  File "e:\microsoft\azurecli\azure-cli\src\azure-cli\azure\cli\command_modules\dls\custom.py", line 304, in set_adls_item_acl_entry
    client.modify_acl_entries(path, acl_spec)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\core.py", line 509, in modify_acl_entries
    self._acl_call('MODIFYACLENTRIES', path, acl_spec, invalidate_cache=True)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\core.py", line 459, in _acl_call
    to_return = self.azure.call(action, posix_path, **parms)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\lib.py", line 454, in call
    self.log_response_and_raise(response, err, level=exception_log_level)
  File "e:\microsoft\azurecli\azure-cli\env\lib\site-packages\azure\datalake\store\lib.py", line 349, in log_response_and_raise
    raise exception
azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, .
akshat-harit commented 4 years ago

@Juliehzl Sorry somehow missed this in my email. Can you tell me how you reproduced this?

vmignot commented 3 years ago

@Juliehzl Sorry somehow missed this in my email. Can you tell me how you reproduced this?

Hi! I too have this issue! You can reproduce it with the CLI : $ az dls fs access set-entry --account testdlsaccount --acl-spec user:${A_USER_ID}:r-w --path "/"

CLIInternalError: The command failed with an unexpected error. Here is the traceback:
Data-lake REST exception: MODIFYACLENTRIES, .
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
    raise ex
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "/usr/local/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 784, in default_command_handler
    return op(**command_args)
  File "/usr/local/lib/python3.6/site-packages/azure/cli/command_modules/dls/custom.py", line 304, in set_adls_item_acl_entry
    client.modify_acl_entries(path, acl_spec)
  File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/core.py", line 509, in modify_acl_entries
    self._acl_call('MODIFYACLENTRIES', path, acl_spec, invalidate_cache=True)
  File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/core.py", line 459, in _acl_call
    to_return = self.azure.call(action, posix_path, **parms)
  File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/lib.py", line 454, in call
    self.log_response_and_raise(response, err, level=exception_log_level)
  File "/usr/local/lib/python3.6/site-packages/azure/datalake/store/lib.py", line 349, in log_response_and_raise
    raise exception
azure.datalake.store.exceptions.DatalakeRESTException: Data-lake REST exception: MODIFYACLENTRIES, .
To open an issue, please run: 'az feedback'

Hope this helps!