search

Azure / aksArc

# Welcome to the Azure Kubernetes Service on Azure Stack HCI repo This is where the AKS-HCI team will track features and issues with AKS-HCI. We will monitor this repo in order to engage with our community and discuss questions, customer scenarios, or feature requests. Checkout our projects tab to see the roadmap for AKS-HCI!
MIT License
109 stars 45 forks source link

[BUG] VM Extension 'ConfigureAksHciHost' failed #356

Open belginceran opened 8 months ago

belginceran commented 8 months ago

Hello,

the deployment getting operation failed error while creating the VM. I am pasting the error below :

{"code":"VMExtensionProvisioningError","message":"VM has reported a failure when processing extension 'ConfigureAksHciHost' (publisher 'Microsoft.Powershell' and type 'DSC'). Error message: \"DSC Configuration 'AKSHCIHost' completed with error(s). Following are the first few: The term 'Choco' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. The PowerShell DSC resource '[cChocoInstaller]InstallChoco' with SourceInfo 'C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.5\DSCWork\akshcihost.0\akshcihost.ps1::621::9::cChocoInstaller' threw one or more non-terminating errors while running the Set-TargetResource functionality. These errors are logged to the ETW channel called Microsoft-Windows-DSC/Operational. Refer to this channel for more details. The SendConfigurationApply function did not succeed. LCM failed to start desired state configuration manually.\". More information on troubleshooting is available at https://aka.ms/VMExtensionDSCWindowsTroubleshoot. "}

Could you please take look into it.

Thanks

Welasco commented 8 months ago

I went ahead and did some troubleshooting today. Looks like Choco is failing to be deployed because of a folder at c:\choco was created prior to the installation step.

I also saw a recent repo change updating Choco for 2.6.0.0 in this commit, but I don't see the file https://github.com/Azure/aks-hybrid/blob/main/eval/dsc/akshcihost.zip been updated which is used a reference during the VM deployment in the ARM template.

Log for reference:

Log file path: C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\2.83.5\DscExtensionHandler.0.20231012-140727.log

VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]: LCM:  [ Start  Resource ]  [[cChocoInstaller]InstallChoco]
VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]: LCM:  [ Start  Test     ]  [[cChocoInstaller]InstallChoco]
VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Test-TargetResource
VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Test-ChocoInstalled
VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Env:Path contains: 
C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32
\OpenSSH\;C:\Program Files\dotnet\
VERBOSE: [2023-10-12 14:11:57Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Test-Command choco
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
choco does NOT exist
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
NO - Choco is not Installed
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Choco is not installed, calling set
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]: LCM:  [ End    Test     ]  [[cChocoInstaller]InstallChoco]  
in 1.6090 seconds.
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]: LCM:  [ Start  Set      ]  [[cChocoInstaller]InstallChoco]
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Start Set-TargetResource
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Performing the operation "Download and Install" on target "Chocolatey".
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Install-Chocolatey
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
[ChocoInstaller] Creating c:\choco
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Setting ChocolateyInstall environment variables
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Env:ChocolateyInstall has c:\choco
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Downloading https://chocolatey.org/install.ps1 to C:\Windows\TEMP\83abf2f1-ccd6-4284-82ef-f3552c038efb\install.ps1
VERBOSE: [2023-10-12 14:11:59Z] [WARNING] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
An existing Chocolatey installation was detected. Installation will not continue. This script will not overwrite 
existing installations.

If there is no Chocolatey installation at 'c:\choco', delete the folder and attempt the installation again.

Please use choco upgrade chocolatey to handle upgrades of Chocolatey itself.

If the existing installation is not functional or a prior installation did not complete, follow these steps:

 - Backup the files at the path listed above so you can restore your previous installation if needed.

 - Remove the existing installation manually.

 - Rerun this installation script.

 - Reinstall any packages previously installed, if needed (refer to the lib folder in the backup).

Once installation is completed, the backup folder is no longer needed and can be deleted.
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Adding Choco to path
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Env:Path has 
C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32
\OpenSSH\;C:\Program Files\dotnet\;c:\choco
VERBOSE: [2023-10-12 14:11:59Z] [ERROR] The term 'Choco' is not recognized as the name of a cmdlet, function, script 
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct 
and try again.
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]:                            [[cChocoInstaller]InstallChoco] 
Finish InstallChoco
VERBOSE: [2023-10-12 14:11:59Z] [VERBOSE] [AKSHCIHost001]: LCM:  [ End    Set      ]  [[cChocoInstaller]InstallChoco]  
in 0.4070 seconds.
VERBOSE: [2023-10-12 14:11:59Z] [ERROR] The PowerShell DSC resource '[cChocoInstaller]InstallChoco' with SourceInfo 
'C:\Packages\Plugins\Microsoft.Powershell.DSC\2.83.5\DSCWork\akshcihost.0\akshcihost.ps1::621::9::cChocoInstaller' 
threw one or more non-terminating errors while running the Set-TargetResource functionality. These errors are logged to
 the ETW channel called Microsoft-Windows-DSC/Operational. Refer to this channel for more details.

Reviewing Chocos installation script install.ps1 it has a section that check the existence of the installation folder which on this case is c:\choco and if it does exist it won't be installed:

function Test-ChocolateyInstalled {
    [CmdletBinding()]
    param()

    $checkPath = if ($env:ChocolateyInstall) { $env:ChocolateyInstall } else { "$env:PROGRAMDATA\chocolatey" }

    if ($Command = Get-Command choco -CommandType Application -ErrorAction Ignore) {
        # choco is on the PATH, assume it's installed
        Write-Warning "'choco' was found at '$($Command.Path)'."
        $true
    }
    elseif (-not (Test-Path $checkPath)) {
        # Install folder doesn't exist
        $false
    }
    else {
        # Install folder exists
        if (Get-ChildItem -Path $checkPath) {
            Write-Warning "Files from a previous installation of Chocolatey were found at '$($CheckPath)'."
        }

        # Return true here to prevent overwriting an existing installation
        $true
    }
}

function Install-7zip {
    [CmdletBinding()]
    param(
        [Parameter(Mandatory = $true)]
        [string]
        $Path,

        [Parameter(Mandatory = $false)]
        [hashtable]
        $ProxyConfiguration
    )
    if (-not (Test-Path ($Path))) {
        Write-Host "Downloading 7-Zip commandline tool prior to extraction."
        Request-File -Url 'https://community.chocolatey.org/7za.exe' -File $Path -ProxyConfiguration $ProxyConfiguration
    }
    else {
        Write-Host "7zip already present, skipping installation."
    }
}

#endregion Functions

#region Pre-check

# Ensure we have all our streams setup correctly, needed for older PSVersions.
Set-PSConsoleWriter

if (Test-ChocolateyInstalled) {
    $message = @(
        "An existing Chocolatey installation was detected. Installation will not continue. This script will not overwrite existing installations."
        "If there is no Chocolatey installation at '$env:ChocolateyInstall', delete the folder and attempt the installation again."
        ""
        "Please use `choco upgrade chocolatey` to handle upgrades of Chocolatey itself."
        "If the existing installation is not functional or a prior installation did not complete, follow these steps:"
        " - Backup the files at the path listed above so you can restore your previous installation if needed."
        " - Remove the existing installation manually."
        " - Rerun this installation script."
        " - Reinstall any packages previously installed, if needed (refer to the `lib` folder in the backup)."
        ""
        "Once installation is completed, the backup folder is no longer needed and can be deleted."
    ) -join [Environment]::NewLine

    Write-Warning $message

    return
}
mkostersitz commented 8 months ago

Thanks folks for reporting. We had to update Choco to the latest version yesterday and that resulted in a break. We are working on a fix

mkostersitz commented 8 months ago

Quick update here. Still working on a fix for this