AAD Linux Signon / az ssh vm: Conditional access policies which require MFA do not work on newer versions of az cli

rybal06 commented 1 year ago

Describe the bug

Enabling MFA in a conditional access policy for AAD VM signin on Linux hosts doesn't work on the latest versions of Az CLI. It works on 2.41.0, but does not work on 2.45.0 or 2.49.0. Did not test other versions.

Steps to repro:

On a MacOS or Windows Machine, install the latest az cli.
Deploy a linux virtual machine and install the AAD login extension.
Grant yourself the Virtual Machine user or virtual machine administrator IAM role.
Verify you can signin to the virtual machine successfully using az ssh vm through az cli.
Enable a conditional access policy to require Microsoft MFA for linux VM signon (per https://learn.microsoft.com/en-us/azure/active-directory/devices/howto-vm-sign-in-azure-ad-linux#enforce-conditional-access-policies)
Attempt to az ssh vm again. Note one of two things happens: A) You are signed into the virtual machine without being prompted for MFA (fails open!) B) A browser window opens to the wrong url (not the MFA challenge, but rather a generic azure ad signin page), and azure cli immediately returns this error message: AADSTS50158: External security challenge not satisfied. User will be redirected to another page or authentication provider to satisfy additional authentication challenges.Trace ID: 62f7d112-2f85-4225-9f07-7b062e714d00Correlation ID: 2739b594-c714-43a8-8573-5e36fd448305Timestamp: 2023-06-23 21:24:51Z
Downgrade the az cli version to 2.41.0 on the Windows or MacOS device.
Try step 6 again. Now you are brought to a page with an MFA challenge and it works as expected.

We have reproduced this on both MacOS and Windows and with Edge, Safari, and Firefox set as default browsers.

Related command

az ssh vm --ip

Errors

AADSTS50158: External security challenge not satisfied. User will be redirected to another page or authentication provider to satisfy additional authentication challenges.Trace ID: Correlation ID: Timestamp: 2023-06-23 21:24:51Z

Issue script & Debug output

az ssh vm --ip 172.27.0.135 --debug cli.knack.cli: Command arguments: ['ssh', 'vm', '--ip', '172.27.0.135', '--debug'] cli.knack.cli: init debug log: Enable color in terminal. cli.knack.cli: Event: Cli.PreExecute [] cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x10174a320>, <function OutputProducer.on_global_arguments at 0x1017f67a0>, <function CLIQuery.on_global_arguments at 0x1018579a0>] cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] cli.azure.cli.core: Modules found from index for 'ssh': ['azext_ssh'] cli.azure.cli.core: Loading command modules: cli.azure.cli.core: Name Load Time Groups Commands cli.azure.cli.core: Total (0) 0.000 0 0 cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next'] cli.azure.cli.core: Loading extensions: cli.azure.cli.core: Name Load Time Groups Commands Directory cli.azure.cli.core: ssh 0.068 1 4 /Users/myusername/.azure/cliextensions/ssh cli.azure.cli.core: Total (1) 0.068 1 4
cli.azure.cli.core: Loaded 1 groups, 4 commands. cli.azure.cli.core: Found a match in the command table. cli.azure.cli.core: Raw command : ssh vm cli.azure.cli.core: Command table: ssh vm cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x1022577f0>] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/Users/myusername/.azure/commands/2023-06-23.16-45-05.ssh_vm.79194.log'. az_command_data_logger: command args: ssh vm --ip {} --debug cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x1022781f0>] cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad [] cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x1022b1c60>, <function register_cache_arguments..add_cache_arguments at 0x1022b1d80>] cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded [] cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [] cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x1017f6830>, <function CLIQuery.handle_query_parameter at 0x101857a30>, <function register_ids_argument..parse_ids_arguments at 0x1022b1cf0>] az_command_data_logger: extension name: ssh az_command_data_logger: extension version: 1.1.6 cli.azure.cli.core.commands.client_factory: Getting management service client client_type=ComputeManagementClient cli.azure.cli.core.auth.persistence: build_persistence: location='/Users/myusername/.azure/msal_token_cache.json', encrypt=False cli.azure.cli.core.auth.binary_cache: load: /Users/myusername/.azure/msal_http_cache.bin urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com//discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com//v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com//kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} msal.application: Broker enabled? False cli.azext_ssh.ssh_utils: Running ssh-keygen command ssh-keygen -f /var/folders/80/t21hqtx533z6jcsvfwkr9pqc0000gp/T/aadsshcert4tb47k36/id_rsa -t rsa -q -N urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None) msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com//discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com//v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com//oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com//kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'} msal.application: Broker enabled? False cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://pas.windows.net/CheckMyAccess/Linux/.default',), kwargs={'data': {'token_type': 'ssh-cert', 'req_cnf': '{"kty": "RSA", "n": "< omitted >", "e": "AQAB", "kid": "< omitted >"}', 'key_id': '< omitted >'}} cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://pas.windows.net/CheckMyAccess/Linux/.default',), claims=None, kwargs={'data': {'token_type': 'ssh-cert', 'req_cnf': '{"kty": "RSA", "n": "< omitted >", "e": "AQAB", "kid": "< omitted >"}', 'key_id': '< omitted >'}} msal.application: Found 1 RTs matching {'environment': 'login.microsoftonline.com', 'home_account_id': '.', 'family_id': '1'} msal.telemetry: Generate or reuse correlation_id: e2590bb3-8913-46d7-aadc-e671ef09d32d msal.application: Cache attempts an RT urllib3.connectionpool: Starting new HTTPS connection (1): login.microsoftonline.com:443 urllib3.connectionpool: https://login.microsoftonline.com:443 "POST //oauth2/v2.0/token HTTP/1.1" 200 5135 msal.token_cache: event={ "client_id": "", "data": { "claims": "{\"access_token\": {\"xms_cc\": {\"values\": [\"CP1\"]}}}", "key_id": "", "refresh_token": "****", "req_cnf": "{\"kty\": \"RSA\", \"n\": \"\"}", "scope": [ "https://pas.windows.net/CheckMyAccess/Linux/.default", "profile", "offline_access", "openid" ], "token_type": "ssh-cert" }, "environment": "login.microsoftonline.com", "grant_type": "refresh_token", "params": null, "response": { "access_token": "****", "client_info": ", "expires_in": 3599, "ext_expires_in": 3599, "foci": "1", "id_token": "****", "scope": "https://pas.windows.net/CheckMyAccess/Linux/user_impersonation https://pas.windows.net/CheckMyAccess/Linux/.default", "token_type": "ssh-cert" }, "scope": [ "https://pas.windows.net/CheckMyAccess/Linux/user_impersonation", "https://pas.windows.net/CheckMyAccess/Linux/.default" ], "skip_account_creation": true, "token_endpoint": "https://login.microsoftonline.com//oauth2/v2.0/token" } cli.azext_ssh.custom: Generating certificate /var/folders/80/t21hqtx533z6jcsvfwkr9pqc0000gp/T/aadsshcert4tb47k36/id_rsa.pub-aadcert.pub cli.azext_ssh.ssh_utils: Running ssh-keygen command ssh-keygen -L -f /var/folders/80/t21hqtx533z6jcsvfwkr9pqc0000gp/T/aadsshcert4tb47k36/id_rsa.pub-aadcert.pub cli.azext_ssh.ssh_utils: Running ssh command ssh 172.27.0.135 -l myusername@.com -i /var/folders/80/t21hqtx533z6jcsvfwkr9pqc0000gp/T/aadsshcert4tb47k36/id_rsa -o CertificateFile="/var/folders/80/t21hqtx533z6jcsvfwkr9pqc0000gp/T/aadsshcert4tb47k36/id_rsa.pub-aadcert.pub" -vvv

Expected behavior

A browser window opens automatically and prompts for MFA as happens when az cli 2.41.0 is installed.

Environment Summary

azure-cli 2.49.0

core 2.49.0 telemetry 1.0.8

Extensions: ssh 1.1.6

Dependencies: msal 1.20.0 azure-mgmt-resource 22.0.0

Additional context

No response

yonzhan commented 1 year ago

Thank you for opening this issue, we will look into it.

navba-MSFT commented 1 year ago

Adding service team to look into this.

rybal06 commented 1 year ago

We can no longer reproduce this issue; but we have not made any changes to the AAD conditional access policies which would have resolved this I am not sure if any fixes were deployed by the Microsoft teams to resolve this; or perhaps there was an intermittent issue, but either way; we can no longer reproduce this issue on Az cli 2.49.0.

GraemeMeyerGT commented 9 months ago

@rybal06 I may be experiencing this as well with Az CLI versions 2.54 and 2.56. I have an open support ticket with MS Azure support to investigate and we're doing some troubleshooting.

Sequence of steps:

Deploy a linux virtual machine and install the AAD login extension. (I'm using Ubuntu 22.04 LTS in the VM)
Grant myself Virtual Machine user or Virtual Machine administrator IAM role.
Verify that I can sign-in to the virtual machine successfully using az ssh vm command through az cli.
Configure Conditional Access Policy with the following settings:
- Name: "Linux VM Sign-In"
- Users: Select Users and Groups --> "$myUserAccount"
- Target resources: Cloud Apps --> Include --> Select Apps --> "Azure Linux VM Sign-In"
- Conditions: Device Platforms --> Include --> "Any Device" (I've tried all devices, just linux, and no conditions applied - all with the same results)
- Access Controls:
- Grant: Grant Access --> "Require multifactor authentication" (I've also tried the new "Require authentication strength")
- Enable Policy: "On"
Wait a few minutes
Use the az ssh vm command to sign into the VM - no authentication step is presented, signs in without any password/MFA/anything

Unless I have misunderstood how conditional access should work/what it should do, it seems clear to me that something is not working here. Not necessarily an Azure CLI issue, but I wanted to make a note of this here as well and see if this matches your experience.

Some troubleshooting notes:

I have now tried this on two separate physical machines, which are joined to two separate domains, none of which are the domain/az tenant the VM is in
I have tried this with the in-browser CloudShell and seen the same results - no authentication step is presented
Sign-in logs are an absolute mess with the migration to the new Preview view, but in general I'm not seeing much evidence of this activity in the sign-in logs, other than the occasional interactive sign-in event when I fail to log into the VM correctly due to having not yet assigned myself the IAM role. Successful signins seem not to be generating logs.

Azure / azure-cli