Set-AzureDeployment NullRefException

jeffyoung commented 9 years ago

Using Azure SDK 2.7, Azure PowerShell 0.9.5

Deploying a Cloud service with Diagnostic Extensions and the Upgrade switch causes a NullRefException. If we don't use Diagnostic Extensions, no NullRefs. If we don't use the Upgrade switch, no NullRefs.

(I've found the same NullRefs with SDK 2.6 and Azure PowerShell 0.9.4. No NullRefs with 0.9.2)

Set-AzureDeployment : Object reference not set to an instance of an object.

... eployment = Set-AzureDeployment -Upgrade -ServiceName $ServiceName -P ...


- CategoryInfo          : NotSpecified: (:) [Set-AzureDeployment], NullReferenceException
- FullyQualifiedErrorId : System.NullReferenceException,Microsoft.WindowsAzure.Commands.ServiceManagement.HostedServices.SetAzureDeploymentCommand

$azureDeployment = Set-AzureDeployment -Upgrade -ServiceName $ServiceName -Package $servicePackageFile -Configuration $serviceConfigFile -Slot $Slot -ExtensionConfiguration $diagnosticExtensions

We have a deployment script we use to deploy Azure Cloud services that determines if a solution has diagnostics extensions. If it does, we build up the array for -ExtensionConfiguration and provide the Upgrade switch if the build specifies it. With those 2 conditions, a NullRef occurs blocking deployment.

I can provide a simpler PS script that reproduces the issue if necessary. The same script works fine with version 0.9.2 of the cmdlets.

eegasai commented 9 years ago

Hi Jeff, I have the same issue with power shell cmdlts in build vNext. Can you share the sample script with me?

Maarten88 commented 9 years ago

We also have this issue. Using Build vNext, after upgrade to Azure SDK 2.7 deployment (which worked fine on 2.5) stopped working with this error:

Set-AzureDeployment -Upgrade -ServiceName ez-dev -Package C:\a\fb58bfdc\EigenZorg\src\EigenZorg.CloudService\bin\Release\app.publish\EigenZorg.CloudService.cspkg -Configuration C:\a\fb58bfdc\EigenZorg\src\EigenZorg.CloudService\bin\Release\app.publish\ServiceConfiguration.Test.cscfg -Slot Production -ExtensionConfiguration <extensions>
Object reference not set to an instance of an object.
******************************************************************************
Finishing task: AzureCloudPowerShellDeployment
******************************************************************************
Task AzureCloudPowerShellDeployment failed. This caused the job to fail. Look at the logs for the task for more details.
******************************************************************************
Finishing Build

tomhreb commented 9 years ago

Hello, same issue here. Using VS 2013, Azure SDK 2.7, Azure Powershell from 07/20 and custom Azure powershell deployment scripts.

I have tried to recreate the wadcfgx files with the SDK 2.7, but did not help.

Thanks Tom

tomhreb commented 9 years ago

Hello, today I've updated both Azure SDK and Powershell to the latest versions, but still the same issue.

My configuration is as follows

2 roles: web role and worker role
enabled plugins: diagnostics, remote desktop on both roles
I am providing config only for the diagnostics plugins, not for remote desktop plugins (I rely on the configuration of this in the cloud config file) -- I think this may be causing the NullReferenceException

Thanks Tom

jseekamp commented 9 years ago

I am having this same exact issue as well. I pasted a section of my deployment script.

Write-Host Setting Subscription...
Set-AzureSubscription -SubscriptionName $subscriptionname -SubscriptionId $sub -Certificate $cert -CurrentStorageAccount $storageaccountname

Write-Host Selecting Subscription...
Select-AzureSubscription -SubscriptionName $subscriptionname

Write-Host Setting the storage account context...
$storageContext = New-AzureStorageContext -StorageAccountName $storageaccountname -StorageAccountKey $storageaccountkey

Write-Host Get diagnostic configuration...
$diagnosticConfig = New-AzureServiceDiagnosticsExtensionConfig -StorageContext $storageContext -DiagnosticsConfigurationPath $diagnosticConfigPath -Role $rolename

Write-Host Getting service...
$deployment = Get-AzureDeployment -Slot Staging -ServiceName $servicename -ErrorAction silentlycontinue

if($deployment.Status -ne $null)
{
    Write-Host Found existing staging deployment
    Write-Host Running in-place upgrade...
    Set-AzureDeployment -Upgrade -Slot Staging -ServiceName $servicename -Package $package -Configuration $config -Label $buildLabel -Mode Auto -Force -ErrorAction Stop -ExtensionConfiguration $diagnosticConfig
}
else
{
    Write-Host Creating new staging deployment...
    New-AzureDeployment -Slot Staging -ServiceName $servicename -Package $package -Configuration $config -Label $buildLabel -ErrorAction Stop -ExtensionConfiguration $diagnosticConfig
}

I receive the following error

Set-AzureDeployment : Object reference not set to an instance of an object.
At 
Artifacts\DeployProduction.ps1:45 char:5
Set-AzureDeployment -Upgrade -Slot Production -ServiceName $servicename -Pac ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Set-AzureDeployment], NullReferenceException
+ FullyQualifiedErrorId : System.NullReferenceException,Microsoft.WindowsAzure.Commands.ServiceManagement.HostedServices.SetAzureDeploymentCommand

This script worked before I added the -ExtensionConfiguration switch

brendangreen commented 9 years ago

Seeing the same problem as everyone else above.

VSO build vNext, my build agent upgraded to Azure SDK 2.7 in conjunction with Azure Powershell 0.9.6.

All deployments of cloud services are now failing with the same error. Log from most recent build:

 2015-08-14T03:37:05.3585066Z New-AzureServiceDiagnosticsExtensionConfig -Role My.Azure.Processor -StorageContext <context> -DiagnosticsConfigurationPath w:\agent\_work\d79d2c17\staging\Extensions\PaaSDiagnostics.My.Azure.Processor.PubConfig.xml
 2015-08-14T03:37:07.8270737Z Get-AzureDeployment -ServiceName my-processor-qa -Slot Production -ErrorAction SilentlyContinue
 2015-08-14T03:37:12.1705485Z Set-AzureDeployment -Upgrade -ServiceName my-processor-qa -Package w:\agent\_work\d79d2c17\staging\My.Azure.cspkg -Configuration w:\agent\_work\d79d2c17\staging\ServiceConfiguration.QA.cscfg -Slot Production -ExtensionConfiguration <extensions>
 2015-08-14T03:37:14.5298165Z ##[error]Object reference not set to an instance of an object.

Prior deployments (Azure SDK 2.4 and Azure Powershell ... 0.9.5 or 0.9.4 not sure which it was) have had no issues.

Has anyone determined a workaround (besides Publishing from Visual Studio)?

mykolad commented 9 years ago

Same problem for me - I'm using "Azure Cloud Service Deployment" step from Visual Studio Online (script Publish-AzureCloudDeployment.ps1 from this repository: https://github.com/Microsoft/vso-agent-tasks). When I set "Allow Upgrade" checkbox, it fails to deploy with message:

2015-08-13T14:39:09.0113687Z ##[error]Object reference not set to an instance of an object.

tomhreb commented 9 years ago

@brendangreen I am deploying the extensions separately, but it takes ages this way. Set-AzureServiceDiagnosticsExtension -ServiceName $ServiceName -Role $RoleName -StorageContext $storageContext -Slot $Slot -DiagnosticsConfigurationPath $DiagnosticConfiguration

ErikCL commented 9 years ago

I'm having the same issue as @mykolad. Using the "Azure Cloud Service Deployment" task in a VSO vNext build I get the same error ( ##[error]Object reference not set to an instance of an object.) when "Allow Upgrade" is checked.

I was also able to verify that @mykolad 's workaround is effective. By unchecking the "Allow Upgrade" checkbox I was able to successfully deploy my Cloud Service.

markcowl commented 9 years ago

@singhkay @huangpf Any idea on this?

hermesramos commented 9 years ago

Samy issue. Microsoft always releasing half tested products (crapware)

jseekamp commented 9 years ago

I found a workaround for this issue.

"ExtensionConfiguration: Specifies an extension that is added to the cloud service. In Azure, a process can run as an extension of a cloud service. You must add an extension to the cloud service by using Add Extension before it can be added to the deployment during an update." https://msdn.microsoft.com/en-us/library/azure/ee460793.aspx

From this I guessed that the powershell command Set-AzureDeployment wasn't checking to see if the extension was already deployed when doing an inplace upgrade. So I changed my script to check first and install before doing the upgrade. If the extension already exists, an upgrade with extension configuration will succeed.

Here is my new deployment script:

    $diagnosticExtension = Get-AzureServiceExtension -ServiceName $servicename -Slot Production -ExtensionName "PaaSDiagnostics"

    if($diagnosticExtension.Extension -ne "PaaSDiagnostics")
    {
        Write-Host Diagnostic extension not found, deploying it...
        Set-AzureServiceDiagnosticsExtension -ServiceName $servicename -Slot Production -Role $rolename -StorageContext $storageContext -DiagnosticsConfigurationPath $diagnosticconfigpath

        Write-Host Running in-place upgrade...
        Set-AzureDeployment -Upgrade -Slot Production -ServiceName $servicename -Package $package -Configuration $config -Label $buildLabel -Mode Auto -Force -ErrorAction Stop
    }
    else
    {
        Write-Host Diagnostic extension found, updating it...

        Write-Host Running in-place upgrade...
        Set-AzureDeployment -Upgrade -Slot Production -ServiceName $servicename -Package $package -Configuration $config -Label $buildLabel -Mode Auto -Force -ErrorAction Stop -ExtensionConfiguration $diagnosticConfig
    }

Of course all of this should be done in the powershell command, but it is a viable workaround for us.

brendangreen commented 9 years ago

@jseekamp @hermesramos this has been fixed in the 0.9.7 release.

eegasai commented 9 years ago

Any update on when is it going to be pushed to production

jseekamp commented 9 years ago

@eegasai Just downloaded the latest 0.9.7 from the Web Platform Installer

devigned commented 9 years ago

Closing the issue since the fix in the 0.9.7 release should resolve this issue. Please reopen if this issue persists.

jameshowe commented 9 years ago

I've just started to hit this problem using the following cmdlets in 0.9.8:

New-AzureSBNamespace Get-AzureSBAuthorizationRule

This was working and then just suddenly started throwing NREs.

huangpf commented 9 years ago

@jameshowe But it seems it's not related to the Set-AzureDeployment cmdlet. Could you please run yours with -Debug and help collect the logs? Thanks.

Azure / azure-powershell

Set-AzureDeployment NullRefException #646