Azure / aksArc

# Welcome to the Azure Kubernetes Service on Azure Stack HCI repo This is where the AKS-HCI team will track features and issues with AKS-HCI. We will monitor this repo in order to engage with our community and discuss questions, customer scenarios, or feature requests. Checkout our projects tab to see the roadmap for AKS-HCI!
MIT License
109 stars 45 forks source link

Set-AksHciRegistration is not saving SPN secret in AzureRmContext.json causes install-AksHci fail #282

Open HassanAlmanasrah opened 1 year ago

HassanAlmanasrah commented 1 year ago

when I run Install-AksHci -verbose its failing with the below error Message : Install-AksHci - Unable to get ServicePrincipalSecret from Azure Context.

the failure is in specific with command Invoke-KvaCtlWithAzureContext , checking the AzureRmContext.json file, it doesn't contain SPN secret any more in it.

AKSHybrid version: 1.0.15.11104

HassanAlmanasrah commented 1 year ago

as workaround, added the SPN secret in the file manually which passed this issue

apluchik commented 1 year ago

@HassanAlmanasrah could you please provide the steps you ran to hit this issue?

HassanAlmanasrah commented 1 year ago

@apluchik here are the steps:

1. $vnet = New-AksHciNetworkSetting -name -vswitchName -vipPoolStart -vipPoolEnd -vlanID

2. Set-AksHciConfig -imageDir $imageDir -workingDir $workingDir -cloudConfigLocation $cloudConfigLocation -vnet $vnet -version "1.0.13.10907" ` -cloudservicecidr

  1. useSPN Set-AksHciRegistration -SubscriptionId $subscription -ResourceGroupName $resource_group -TenantId $tenantID -Credential $credential

  2. $VerbosePreference = "Continue" Install-AksHci -verbose

Step 3 is going to create AzureRmContext.json file in path c:\users\user.Azure usually this file contains both SPN and its secret , which will be used during installation in step 4. Recently, this file doesn't contain "ServicePrincipalSecret", which causes deployment failure. As a workaround, When I added the SPN secret manually , it passes the failed step. So, my explanation that command Set-AksHciRegistration is not saving the secret in the generated file like in the past, could be newer version for the module was released recently?

cblackuk commented 1 year ago

Having the exact same issue:

Unable to get ServicePrincipalSecret from Azure Context.

TestName       : Validate KVA
Category       : KVA
TestResult     : Failed
Details        : Unable to get ServicePrincipalSecret from Azure Context.
Recommendation :

Steps to repro:

$TenantId = "xxxx"
$Sub = "xxxx"
$RG = "xxx"
$SPNClientId = "xxxx"
$SPNClientSecret = "xxxx"
[SecureString] $SPNClientSecretSecure = ConvertTo-SecureString $SPNClientSecret -AsPlainText -Force

$Creds = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $SPNClientId, $SPNClientSecretSecure

Set-AksHciRegistration -SubscriptionId $Sub -TenantId $TenantId -ResourceGroupName $RG -Credential $Creds

Also tried with -SkipLogin and -AccountId, same result

cblackuk commented 1 year ago

ServicePrincipalSecret

Update:

To fix it on my system - we had to remove Az.Accounts - for some reason 2.11.2 and 2.10.2 were present.

Run that twice and it works now :-)

Get-Module -Name "Az.Accounts" -ListAvailable | select -First 1 | Uninstall-Module -Force 

image

cblackuk commented 1 year ago

Update: Better fix :-)

Import-Module -Name "Az.Accounts" -RequiredVersion 2.6.0

Also, the reason why 2.11.2 is there is because it is one of the dependencies for https://www.powershellgallery.com/packages/Az.StackHCI/1.4.2 which most likely already installed on the HCI node :)

Dependencies Az.Accounts (>= 2.11.2)

Hopefully that helps somebody :-)

shobhit-kumar-code commented 1 year ago

Thanks @cblackuk . I have been following your comments over the past few days which helped me past my troubles. I was planning to put the latest comment similar to yours, but it is already there :)

cblackuk commented 1 year ago

Thanks @cblackuk . I have been following your comments over the past few days which helped me past my troubles. I was planning to put the latest comment similar to yours, but it is already there :)

Glad it helped :-)

mbrat2005 commented 1 year ago

I couldn't get this to work with just Import-Module -Name "Az.Accounts" -RequiredVersion 2.6.0 for some reason-- ~/.Azure/AzureRMContext.json still did not have the ServicePrincipalSecret property.

Here are the steps I followed to get this working:

  1. Connect-AzAccount -ServicePrincipal -Credential $spCred -Tenant $tenantId
  2. Modify ~/.Azure/AzureRMContext.json, adding the ServicePrincipalSecret property to the first Account -> ExtendedProperties object within the Contexts object.
  3. Set-AksHciRegistration -subscriptionId $subscription -resourceGroupName $Resource_Group -TenantId $tenantId -Credential $spCred -SkipLogin
igomaa commented 1 year ago

I have tried what @mbrat2005 found, and worked , with Az.Accounts version 2.12.1 only .

mrhoads commented 1 year ago

I wanted to add that I also was able to get this to work only after following what @mbrat2005 described with adding the ServicePrincipalSecret under the account's ExtendedProperties object

cblackuk commented 1 year ago

I wonder if something has changed? Was a new module release or what is the delta? :-)

I think I will be looking at it again in a few weeks, so I guess I will find out :-)

BTW Can you paste a dummy JSON here where you are adding the Secret, please? @mrhoads

cblackuk commented 1 year ago

I see now - the https://www.powershellgallery.com/packages/Az.StackHCI/1.4.3 has changed and the dependency is now: Dependencies Az.Accounts (>= 2.12.0)

Also https://www.powershellgallery.com/packages/AksHci/1.1.68 has been bumped a bunch of times... and yet this is not fixed ;-)

mrhoads commented 1 year ago

It's the line "ServicePrincipalSecret": "" that I manually added

{
  "DefaultContextKey": "my context info here",
  "EnvironmentTable": {},
  "Contexts": {
    "<my subscription name etc.": {
      "Account": {
        "Id": "<my id>",
        "Credential": null,
        "Type": "ServicePrincipal",
        "TenantMap": {},
        "ExtendedProperties": {
          "Subscriptions": "<subscription id>",
          "Tenants": "<tenant id>",
          "ServicePrincipalSecret": "<manually added the SP secret>"

        }
      },`
cblackuk commented 1 year ago

It's the line "ServicePrincipalSecret": "" that I manually added

{
  "DefaultContextKey": "my context info here",
  "EnvironmentTable": {},
  "Contexts": {
    "<my subscription name etc.": {
      "Account": {
        "Id": "<my id>",
        "Credential": null,
        "Type": "ServicePrincipal",
        "TenantMap": {},
        "ExtendedProperties": {
          "Subscriptions": "<subscription id>",
          "Tenants": "<tenant id>",
          "ServicePrincipalSecret": "<manually added the SP secret>"

        }
      },`

Thank you :-)