IISWebAppManagementOnMachineGroupV0 can't detect existing SSL bindings on Server 2022

[altwohill]

I have the following in my yaml pipeline:

  - task: IISWebAppManagementOnMachineGroup@0
    displayName: 'Deploy base website with binding: ${{ binding }}'
    inputs:
      IISDeploymentType: 'IISWebsite'
      ActionIISWebsite: 'CreateOrUpdateWebsite'
#... snipped 
      AddBinding: true
      Protocol: 'https'
      IPAddress: '*'
      Port: 443
      ServerNameIndication: true
      HostNameWithSNI: '${{ binding }}'

It works the first time but fails on repeat deployments on Server 2022 machines.

The issue doesn't occur on our Server 2019 and older machines.

I think the issue is due to some additional lines in the netsh output

Looking at https://github.com/microsoft/azure-pipelines-extensions/blob/78bf251236c84c4da622c125764b7e7434e98a2a/TaskModules/powershell/TaskModuleIISManageUtility/AppCmdOnTargetMachines.ps1#L186 and https://github.com/microsoft/azure-pipelines-extensions/blob/78bf251236c84c4da622c125764b7e7434e98a2a/TaskModules/powershell/TaskModuleIISManageUtility/AppCmdOnTargetMachines.ps1#L201 I see the script is checking specific line numbers before proceeding.

On Server 2022 the checks always fail so it tries to add a new binding, which fails with the error

SSL Certificate add failed, Error: 183
Cannot create a file when that file already exists.

##[error]Process 'netsh' exited with code '1'.

[altwohill]

Ah, I see #993 would fix this

[danspam]

Just spent hours trying to figure out why the IIS web app manage task was failing. Totally broken on Server 2022

[jiggybyte]

This is still completely broken on Server 2022. Such a shame, there's a simple fix to the script (or just don't use hard coded lines, sheesh)

[saephraim]

I just ran into this same issue today. We have installed Windows Server 2022 to migrate sites to. This worked fine in Server 2012 R2.

[jabteles]

+1 preventing to migrate to Server 2022. No more workarounds.

[jbartlettii]

Seems to still be an issue, not really a viable workaround, but you can manually configure the cert or deploy the pipeline the first time with the cert, and then blow away the entire binding and the task shouldn't remove it afterwards so that code deploys and application updates still function but the binds, or certs will not. Just in case you are stuck on 2022 for whatever reason.

[rickjames961]

+1 preventing to migrate to Server 2022.

[LeftTwixWand]

Hi, we're working on it.

[mrunks]

I too am experiencing this problem. Is there any updates or ETA when a possible fix will be released ?

[LeftTwixWand]

Hey @mrunks my colleague is finished the development part and I hope to see the fix released in upcoming days.

[v-schhabra]

Hello @mrunks The mentioned issue has been fixed in this PR https://github.com/microsoft/azure-pipelines-extensions/pull/1220 Could you please use the IISWebAppManagementV3 version of the task and test if it is working fine?

[danielgreen]

@v-schhabra How do we select that version of the task in a classic release pipeline? The Task Version dropdown only has an entry for "0.*".

[LeftTwixWand]

Hey @danielgreen I don't think that it's possible to choose the pipeline version other than the major one. But why do you need to do it?

[danielgreen]

@LeftTwixWand Perhaps I've misunderstood the last post from @v-schhabra.

The problem reported by this issue relates to the IISWebAppManagementOnMachineGroupV0 task, which is what you get when you select the "IIS web app manage" task in DevOps.

When @v-schhabra said "please use the IISWebAppManagementV3 version of the task" - I thought that meant selecting that version in the pipeline somehow.

What are the steps we should take to test the fix in a classic release pipeline?

[ImperatorRuscal]

I'd like to second @danielgreen with the instruction request. I have several pipelines that are continually failing to deploy after an upgrade to WinSvr22 because of this. Any site that is internal-only w/o a SSL binding still works just fine. But redeploying/update-deploying an already running site with an active SSL binding (even the exact same binding as is already active) is causing failures.

I suppose I could technically put in a manual cmd task to force-remove the SSL bindings immediately prior to the Web App Manage task; but that feels like a sub-optimal solution.

How do we get the classic pipeline to use the V3 task?

[v-schhabra]

Hi @ImperatorRuscal, When you add IISWebAppMgmt task in classic release you will be getting the dropdown to select the task version. From there you can choose the V3 task version.

[danielgreen]

Hi @ImperatorRuscal, When you add IISWebAppMgmt task in classic release you will be getting the dropdown to select the task version. From there you can choose the V3 task version.

@v-schhabra

At the moment we still only see 0.* in the Task Version dropdown for that task. Will V3 appear there, if so do we know when?

[v-schhabra]

Hi @danielgreen, You have added another task. I was mentioning regarding this task

[danielgreen]

@v-schhabra I wasn't aware of the WinRM extension tasks for managing IIS. Looks like I should log the same issue in the Azure Pipelines Tasks repo.

The OP mentioned the IISWebAppManagementOnMachineGroupV0 task which has the same name as the task I referred to, and exhibits the same issue.

https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/iisweb-app-management-on-machine-group-v0?view=azure-pipelines

[v-schhabra]

@danielgreen Could you please try to use IISWebAppManagementOnMachineGroupV0 task and share the debugged logs of the pipeline? Thanks.

[danielgreen]

@v-schhabra Sorry, I don't understand what you are asking me to do. I am using the Azure Pipelines task I posted above (not the WinRM task).

The info box for the task I am using links to https://aka.ms/iis-webapp-management-readme, which redirects to https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/IISWebAppManagementOnMachineGroupV0/README.md

The error I see in the logs is the same as in the OP: SSL Certificate add failed, Error: 183 Cannot create a file when that file already exists.

[v-schhabra]

@danielgreen Could you pls try to run WinRM task and add the variable "System.debug" to "true" in the pipeline and share the pipeline logs.

[ImperatorRuscal]

I suppose I could technically put in a manual cmd task to force-remove the SSL bindings immediately prior to the Web App Manage task; but that feels like a sub-optimal solution.

For anyone who cares, and seeing as it appears that nobody is taking this problem seriously enough to actually look into/fix the underlying issue, I did break down and create a drop-in step that can be executed prior to Web App Manage that will "solve" the problem. (It removes any existing SSL cert bindings that match the ones you're about to push with Web App Manage.)

The script assumes that your bindings are in the parameter "Bindings" (which is 1 obvious, and 2 the name used in the template "IIS web site deployment" stage).

Just drop a PowerShell step in prior to the Web App Manage step, and feed it this one-liner (ConvertFrom-Json '$(Parameters.Bindings)').bindings|?{$_.protocol.Trim().toLower() -eq 'https'}|%{Start-Process -FilePath "$env:SystemRoot\System32\netsh.exe" -ArgumentList ('http','delete','sslcert',"hostnameport=$($_.hostname):$($_.port)")} I have been using an errorAction of silentlyContinue just to make sure that this step doesn't stop the release in the event that there aren't existing SSL cert bindings. (Though any netsh errors should be hidden behind the Start-Process's output object.)

Or, if you want the full YAML, then you can use this

steps:
- powershell: '(ConvertFrom-Json ''$(Parameters.Bindings)'').bindings|?{$_.protocol.Trim().toLower() -eq ''https''}|%{Start-Process -FilePath "$env:SystemRoot\System32\netsh.exe" -ArgumentList (''http'',''delete'',''sslcert'',"hostnameport=$($_.hostname):$($_.port)")}'
  errorActionPreference: silentlyContinue
  showWarnings: true
  ignoreLASTEXITCODE: true
  displayName: 'Remove SSL Certificate Bindings (so Web App Manage works properly)'
  continueOnError: true

I greatly dislike that this seems to be required. But at least it has my Release pipes back in operation.

[LeftTwixWand]

Hi @ImperatorRuscal, I apologize for such a big delay in responses. Fo any question please, ping @v-schhabra who's currently working on it. From my side I just providing help with proper environment setup.

[jiggybyte]

This is laughable that this issue is still ongoing 6 months after being reported. There's a simple fix to this, and several people have posted solutions and workarounds here. Incredible that Microsoft provides these tasks in such a broken state, and continually releases things that are so clearly untested.

[ImperatorRuscal]

This is laughable that this issue is still ongoing 6 months after being reported. There's a simple fix to this, and several people have posted solutions and workarounds here. Incredible that Microsoft provides these tasks in such a broken state, and continually releases things that are so clearly untested.

More like 30 months. This particular issue was opened in November of 2021. The truly disheartening part is that this issue was opened with such a detailed notation of what the problem was and where it came from, that it should have been easily corrected by now. Heck, there was a PR from October of 2021 (before this specific issue was even opened) that would have corrected the problem.

The community did a spot-on job of identifying the problem, and then correcting it by modifying the source code and then submitting a Pull Request to bring that fix back into the main branch. Literally the perfect response with zero code overhead needed by the maintainers. But it feels like nobody is actually maintaining the code -- all they had to do was a review and pull... 2 years ago. If they really wanted to be sticklers on convention, then comment on that AND correct it so the PR can be completed. But leaving a PR that fixes a systemic problem to sit unaddressed for 31 months is just silly -- if you don't want to approve it, then give it the decency of a clean execution. And even if you don't use the PR, the answer to this problem (which, again, will affect ALL agents running on Server 2022) has clearly been given, just incorporate it into the repo

That's my rant, and I apologize for anyone who got this in their mailbox without wanting to hear my long windedness. I know that @LeftTwixWand will say they're just helping with environment setup and we should be bugging @v-schhabra -- but this issue is listed as assigned to @LeftTwixWand, and all we want is for someone (anyone, really) to just merge in one of the fixes that the community has already identified.

[b3go]

@danielgreen Could you pls try to run WinRM task and add the variable "System.debug" to "true" in the pipeline and share the pipeline logs.

@v-schhabra the WinRM task is no help, because it doesn't work (can't even be selected) with deployment groups. The WinRM task has to be run in a normal "agent job". IISWebAppManagementOnMachineGroup@0 is the respective task to use in "deployment group jobs". For this to work again on deployment groups you unfortunately also need to fix that issue in the codebase for the IISWebAppManagementOnMachineGroup@0 task not the IISWebAppMgmt@3.