webprofusion / certify

Professional ACME Client for Windows. Certificate Management UI, powered by Let's Encrypt and compatible with all ACME v2 CAs. Download from certifytheweb.com
https://certifytheweb.com
Other
1.49k stars 257 forks source link

Windows Server Essentials 2016 Access Anywhere Certificate renewal script #519

Open SergeCaron opened 4 years ago

SergeCaron commented 4 years ago

I see various complaints regarding WSE 2016 and certificate renewals.

I have been using the script below for years without issues. You have to be patient it takes at least two minutes for everything to restart but it does work.

Regards,

Enable certificate for RDP Gateway

param($result)

Import-Module RemoteDesktopServices

Apply certificate

Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

Restart Network Policy Server, TSGateway, and SSTP Protocol

Restart-Service IAS -Force -ErrorAction Stop Restart-Service TSGateway -Force -ErrorAction Stop Restart-Service SSTPSvc -Force -ErrorAction Stop Write-Host "Done!"

webprofusion-chrisc commented 4 years ago

Thanks is there a complaint somewhere we should be addressing?

We also have a built in Deployment Task for this but it currently only does this, are the other service restarts essential?:


param($result, [switch] $restartServices = $false)

Import-Module RemoteDesktopServices

# Apply certificate
Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

# Optionally restart TSGateway

if ($restartServices -eq $true)
{
    Restart-Service TSGateway -Force -ErrorAction Stop
}
webprofusion-chrisc commented 4 years ago

As an aside you can also now use our built in Deployment Task to stop/start/restart services, so scripting isn't the only way to do it.

SergeCaron commented 4 years ago

Hello Christopher,

On Windows Server 2016 Essentials, if you don’t restart Network Policy Server, VPNs do connect but here is no traffic allowed on the network.

For people using the SSTP protocol, you must restart the SSTVSvc service AFTER restarting the TSGateway.

Note that there is a significant delay during these restarts: on a lightly loaded server, expect a two minutes delay.

Regards,

Serge Caron

SergeCaron commented 4 years ago

Hello Christopher,

I am not certain how to control the exact restart sequence using separate tasks.

Regards,

Serge Caron

webprofusion-chrisc commented 4 years ago

Thanks Serge. Regarding Tasks, they are executed in the order that they appear on the list and you can drag/drop them to re-order.

SergeCaron commented 4 years ago

Hello Christopher,

Yesterday was too hectic to reply.

The task “Deploy to RAS (Direct Access, …” does not document which services are restarted and in what order.

Over the years, I have not seen a recommendation to restart IAS, the now “Network Policy Server”, in he various forums concerned with Windows Server 2016 Essentials.

The same goes for SSTPSvc.

So, does your “Deploy to RAS …” task restart these services, in which order, and does it do anything else ?

Kind regards,

Serge Caron

webprofusion-chrisc commented 4 years ago

Hi Serge, no currently it only runs the following, but I'm happy to update it. Ideally the script would work for everyone and could be combined with other actions for users who need to to do more:

Restart-Service TSGateway -Force -ErrorAction Stop
SergeCaron commented 4 years ago

Hello Christopher,

This does not work for WS2016 Essentials.

Working backwards so you have a better understanding:

  1. SSTPSvc relies on the certificate in use by TSGateway to accept SSTP VPN connections. If this service is not restarted following a TSGateway update, incoming VPN connections will succeed but will dropout almost immediately because of the lack of SSL/TLS support resulting from a certificate mismatch.

  2. TSGateway will honor incoming connections but the remote end will not pass traffic: you can’t even ping the WS2016 server even if you have a valid IP provided by the TSGateway/SSTPSvc pair.

  3. Internet Authentication Service (IAS) was renamed Network Policy Server (NPS) with the introduction of Windows Server 2008. This is the service responsible for allowing VPN traffic on the local network.

Please note that restarting IAS is a long process and you will find many lines in your log. On a less powerful machine, you are looking at two minutes overall delay.

I am presently testing a replacement server and I will gladly use your new functionality if it does the above. Again, please note that I had added a <Write-Host “Done!” > at the end of my script in the hope that it would be part of the log ;-) Perhaps your version could add this simple safeguard.

Kind regards,

Serge Caron

SergeCaron commented 4 years ago

Hello Christopher,

Yesterday, I came across a client configuration where the static script did not work. Our friends at Microsoft are somewhat creative…

Below is a script that will recursively walk the dependencies for the “Network Policy Server” and the “Secure Socket Tunneling Protocol (SSTP) service” : note that these two are not directly or indirectly related, one more of Microsoft’s idiosyncrasies.

Kind regards,

Serge Caron

PS: this is a preliminary version with little or no validation.

param($result)

Import-Module RemoteDesktopServices

Apply certificate

Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

Restart services required for "Access Anywhere"

$global:ServicesToStart = @()

Function CollectDependent($TargetService)

{ $wmidependents = (get-service $TargetService).dependentservices

 $wmidependentservices = Get-WmiObject Win32_Service | Select-object name,state,startmode | where {$wmidependents.name -contains $_.name}

 # Write-Host $TargetService

 foreach ($service in $wmidependentservices){
      if($service.startmode -eq "auto" -or $service.status -eq "Running"){
            # Write-Host "-> $($service.name)"
            # Caution: no effort done to prevent circular definitions
            CollectDependent($service.name)
      }
      else{
            Write-Host "Omitting $($service.name) : service is $($service.state) with the startmode: $($service.startmode)"
      }
 }

 stop-service $TargetService -ErrorAction SilentlyContinue

 $global:ServicesToStart += $TargetService

}

CollectDependent("IAS")

CollectDependent("SSTPSvc")

Write-Host "----"

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Host "Done!"

webprofusion-chrisc commented 4 years ago

Thanks Serge, this is quite extensive, I think I'd be tempted to leave the exercise of stopping and starting related services to the user (i.e they would specify tasks to restart the correct services) as clearly it depends on their environment to some extent.

SergeCaron commented 4 years ago

Hello Christopher,

Indeed, it is quite extensive.

The issue is that you don’t know for sure which services are running (i.e. some services start-up are set to “Manual” and have dependencies of their own) AND you don’t know the VPN configuration in use. Some dependent services are not in use in every installation.

So the user must research the exact sequence of services they need to stop and start : a simple restart of TSGateway will not do. And this depends on the “configuration du jour” ;-). As you can see in the code, TSGateway is not explicitly named: it is a dependent service of the Network Policy Server.

I am going to add a little protection against circular dependencies and I will live with this for a while.

Kind regards,

Serge Caron

De : Christopher Cook notifications@github.com Envoyé : 18 août 2020 23:43 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Thanks Serge, this is quite extensive, I think I'd be tempted to leave the exercise of stopping and starting related services to the user (i.e they would specify tasks to restart the correct services) as clearly it depends on their environment to some extent.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-675834004, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4AOXQ6IDZ4FDEIBQBDSBNC5FANCNFSM4PYY2OMQ.

SergeCaron commented 4 years ago

Hello Christopher,

Production script below : enjoy ;-)

Regards,

Serge Caron

PS : I have changed “Write-Host” to “Write-Warning” so these actions can be seen in the Certify The Web log file.

**

Revision date: 2020.08.19

Copyright (c) 2020 PC-Évolution enr.

This code is licensed under the GNU General Public License (GPL).

THIS CODE IS PROVIDED AS IS WITHOUT WARRANTY OF

ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY

IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR

PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

**

Certify The Web Deployment Script for Windows Server 2016 Essentials (Your Mileage May Vary ;-)

Restart the "Network Policy Server" as well as the SSTP protocol : all dependent services are

stopped and started in the exact reverse order they were stopped.

Note: the Write-Warning is used so that messages will appear in the Certify The Web log file.

param($result)

Import-Module RemoteDesktopServices

Apply certificate

Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

Restart services required for "Access Anywhere"

$global:ServicesToStart = @()

Function CollectDependent($TargetService)

{

Caution: minimal effort done to prevent circular definitions

 if ($global:ServicesToStart -match "$service.name") {
            Write-Warning "Caution! Service $service.name is involved in a circular definition."
      }
 else {
      $wmidependents = (get-service $TargetService).dependentservices

      $wmidependentservices = Get-WmiObject Win32_Service | Select-object name,state,startmode | where {$wmidependents.name -contains $_.name}

      # Write-Host $TargetService

      foreach ($service in $wmidependentservices){
            if($service.startmode -eq "auto" -or $service.status -eq "Running"){
                 # Write-Host "-> $($service.name)"
                 CollectDependent($service.name)
            }
            else{
                 Write-Warning "Omitting $($service.name) : service is $($service.state) with the startmode: $($service.startmode)"
            }
      }

      stop-service $TargetService -ErrorAction SilentlyContinue

      $global:ServicesToStart += $TargetService
 }

}

CollectDependent("IAS")

CollectDependent("SSTPSvc")

Write-Host "----"

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Warning "Done!"

De : Serge Caron Envoyé : 19 août 2020 05:50 À : 'webprofusion/certify' reply@reply.github.com Objet : RE: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hello Christopher,

Indeed, it is quite extensive.

The issue is that you don’t know for sure which services are running (i.e. some services start-up are set to “Manual” and have dependencies of their own) AND you don’t know the VPN configuration in use. Some dependent services are not in use in every installation.

So the user must research the exact sequence of services they need to stop and start : a simple restart of TSGateway will not do. And this depends on the “configuration du jour” ;-). As you can see in the code, TSGateway is not explicitly named: it is a dependent service of the Network Policy Server.

I am going to add a little protection against circular dependencies and I will live with this for a while.

Kind regards,

Serge Caron

De : Christopher Cook notifications@github.com<mailto:notifications@github.com> Envoyé : 18 août 2020 23:43 À : webprofusion/certify certify@noreply.github.com<mailto:certify@noreply.github.com> Cc : Serge Caron SCaron@pcevolution.com<mailto:SCaron@pcevolution.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Thanks Serge, this is quite extensive, I think I'd be tempted to leave the exercise of stopping and starting related services to the user (i.e they would specify tasks to restart the correct services) as clearly it depends on their environment to some extent.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-675834004, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4AOXQ6IDZ4FDEIBQBDSBNC5FANCNFSM4PYY2OMQ.

devast8tor commented 4 years ago

Just wanted to drop a short note to say a huge thank you for Serge for writing and sharing this script. I've successfully deployed this on my personal WSE 2016 and my Access Anywhere install is now running the current cert (instead of the original one I deployed manually). I'll keep an eye on this thread in case additional improvements to the script are added.

SergeCaron commented 3 years ago

Hello Christopher at al.,

There are hidden dependencies between the SSTPsvc and IAS services or there is a race condition of some sort between these two on faster servers.

This is a small revision where declared dependencies for these two services are enumerated SSTPSvc first, IAS second.

The renewal log now shows the exact order in which services are restarted. IAS will now start before SSTPSvc, provided there is no other black magic involved.

Regards,

Serge Caron

2020-12-20 12:09:18.836 -05:00 [INF] Executing command via PowerShell 2020-12-20 12:09:29.884 -05:00 [INF] Omitting RasMan : service is Stopped with the startmode: Disabled Omitting rqs : service is Stopped with the startmode: Manual Attente de l’arrêt du service « Routage et accès distant (RemoteAccess) »… Omitting rqs : service is Stopped with the startmode: Manual Waiting for powershell to complete..5s Attente de l’arrêt du service « Network Policy Server (IAS) »… Attente de l’arrêt du service « Network Policy Server (IAS) »… Restarting services in the following order: IAS TSGateway SSTPSvc RemoteAccess Waiting for powershell to complete..10s Done!

2020-12-20 12:09:29.884 -05:00 [INF] Run Powershell Script :: Task Completed OK

De : devast8tor notifications@github.com Envoyé : 24 septembre 2020 01:11 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Just wanted to drop a short note to say a huge thank you for Serge for writing and sharing this script. I've successfully deployed this on my personal WSE 2016 and my Access Anywhere install is now running the current cert (instead of the original one I deployed manually). I'll keep an eye on this thread in case additional improvements to the script are added.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-698115977, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4EO7TUADCZMYPLVVHDSHLIG5ANCNFSM4PYY2OMQ.

devast8tor commented 3 years ago

Serge,

Did you post your revised code in this thread or as a commit to the project? Looking to test your updated code as the service dependencies have been causing issues for the Remote Desktop Gateway cert application (it keeps holding on to the prior cert instead of applying the latest renewal).

SergeCaron commented 3 years ago

Merry Christmas to All ;-)

The revised code was attached to my previous email : I was not intelligent enough to update the GitHub project … shame on me!

The “revision” is simply changing the order in which the dependencies are enumerated: SSTPSvc is enumerated before IAS and the side effect is that IAS is restarted before any of the other services. Microsoft does not enumerate IAS as a dependency for SSTPSvc, so the point at which it picks up the new certificate is unknown.

As a simple test, here is a scenario where the same certificate is issued at a 15 minutes interval.

So, there is a small bug in the naming of the “friendly name” of the certificate, the “to” part is one hour later probably as the result of some daily savings time discrepancy.

However, the SSTPSvc service picks up the new certificate on all the servers on which this was tested. On December 30th, I still have 5 different installations to update to CTW 5.2.1 and I will report if all is fine with those as well.

Christopher may have fix the naming bug by then ;-)

Kind regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 24 décembre 2020 01:04 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Did you post your revised code in this thread or as a commit to the project? Looking to test your updated code as the service dependencies have been causing issues for the Remote Desktop Gateway cert application (it keeps holding on to the prior cert instead of applying the latest renewal).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-750759431, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4GHAKNCK5FM4OT77JLSWLKULANCNFSM4PYY2OMQ.

devast8tor commented 3 years ago

Merry Christmas to you too! I must be missing something here, because your email attachment didn't seem to make it into the comment thread. Perhaps Chris is able to see it as the project administrator though...

Can you repost the updated code via an in line script block?

On Thu, Dec 24, 2020, 8:41 AM SergeCaron notifications@github.com wrote:

Merry Christmas to All ;-)

The revised code was attached to my previous email : I was not intelligent enough to update the GitHub project … shame on me!

The “revision” is simply changing the order in which the dependencies are enumerated: SSTPSvc is enumerated before IAS and the side effect is that IAS is restarted before any of the other services. Microsoft does not enumerate IAS as a dependency for SSTPSvc, so the point at which it picks up the new certificate is unknown.

As a simple test, here is a scenario where the same certificate is issued at a 15 minutes interval.

  • The ACTIVE certificate (as displayed in a web browser) is valid from “‎20 ‎décembre ‎2020 11:09:15 to ‎20 ‎mars ‎2021 11:09:15”;

  • The first invocation of the renewal process shows In the Certify The Web log file : “2020-12-20 11:55:47.942 -05:00 [INF] Task [Run Powershell Script] :: Task is enabled and primary request was successful.” In the MMC Certificate console, the “friendly name” of the certificate is “passerelle.[secretdomain].com [Certify] - 2020-12-20 10:55:44 to 2021-03-20 11:55:44”

  • The second invocation of the renewal process shows In the Certify The Web log file : “2020-12-20 12:09:18.836 -05:00 [INF] Task [Run Powershell Script] :: Task is enabled and primary request was successful.” In the MMC Certificate console, the “friendly name” of the second certificate is “passerelle.[secretdomain].com [Certify] - 2020-12-20 11:09:15 to 2021-03-20 12:09:15”

So, there is a small bug in the naming of the “friendly name” of the certificate, the “to” part is one hour later probably as the result of some daily savings time discrepancy.

However, the SSTPSvc service picks up the new certificate on all the servers on which this was tested. On December 30th, I still have 5 different installations to update to CTW 5.2.1 and I will report if all is fine with those as well.

Christopher may have fix the naming bug by then ;-)

Kind regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 24 décembre 2020 01:04 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author < author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Did you post your revised code in this thread or as a commit to the project? Looking to test your updated code as the service dependencies have been causing issues for the Remote Desktop Gateway cert application (it keeps holding on to the prior cert instead of applying the latest renewal).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-750759431>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4GHAKNCK5FM4OT77JLSWLKULANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-750924580, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSHEF6V2F5S4Q7KJQVTSWNVLZANCNFSM4PYY2OMQ .

latinkreationz commented 3 years ago

Serge,

Are you able to post your most updated script?

SergeCaron commented 3 years ago

Here is the production script:

**

Revision date: 2020.12.20

Copyright (c) 2020 PC-Évolution enr.

This code is licensed under the GNU General Public License (GPL).

THIS CODE IS PROVIDED AS IS WITHOUT WARRANTY OF

ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY

IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR

PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

**

Certify The Web Deployment Script for Windows Server 2016 Essentials (Your Mileage May Vary ;-)

Restart the "Network Policy Server" as well as the SSTP protocol : all dependent services are

stopped and started in the exact reverse order they were stopped.

Note: the Write-Warning is used so that messages will appear in the Certify The Web log file.

2020.08.19 : Initial version

2020.12.20 : Change order of CollectDependant calls to addres some hidden dependencies.

param($result)

Import-Module RemoteDesktopServices

Apply certificate

Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

Restart services required for "Access Anywhere"

$global:ServicesToStart = @()

Function CollectDependent($TargetService)

{

Caution: minimal effort done to prevent circular definitions

if ($global:ServicesToStart -match "$service.name") {
        Write-Warning "Caution! Service $service.name is involved in a circular definition."
    }
else {
    $wmidependents = (get-service $TargetService).dependentservices

    $wmidependentservices = Get-WmiObject Win32_Service | Select-object name,state,startmode | where {$wmidependents.name -contains $_.name}

    # Write-Host $TargetService

    foreach ($service in $wmidependentservices){
        if($service.startmode -eq "auto" -or $service.status -eq "Running"){
            # Write-Host "-> $($service.name)"
            CollectDependent($service.name)
        } 
        else{
            Write-Warning "Omitting $($service.name) : service is $($service.state) with the startmode: $($service.startmode)"
        }
    }

    stop-service $TargetService -ErrorAction SilentlyContinue

    $global:ServicesToStart += $TargetService
}

}

CollectDependent("SSTPSvc")

CollectDependent("IAS")

Write-Warning "Restarting services in the following order: " foreach($service in $global:ServicesToStart ) { Write-Warning $service }

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Warning "Done!"

SergeCaron commented 3 years ago

(I have issues with this #!$ browser) Here it is again:

**

Revision date: 2020.12.20

Copyright (c) 2020 PC-Évolution enr.

This code is licensed under the GNU General Public License (GPL).

THIS CODE IS PROVIDED AS IS WITHOUT WARRANTY OF

ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY

IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR

PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.

**

Certify The Web Deployment Script for Windows Server 2016 Essentials (Your Mileage May Vary ;-)

Restart the "Network Policy Server" as well as the SSTP protocol : all dependent services are

stopped and started in the exact reverse order they were stopped.

Note: the Write-Warning is used so that messages will appear in the Certify The Web log file.

2020.08.19 : Initial version

2020.12.20 : Change order of CollectDependant calls to addres some hidden dependencies.

param($result)

Import-Module RemoteDesktopServices

Apply certificate

Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

Restart services required for "Access Anywhere"

$global:ServicesToStart = @()

Function CollectDependent($TargetService)

{

Caution: minimal effort done to prevent circular definitions

if ($global:ServicesToStart -match "$service.name") {
        Write-Warning "Caution! Service $service.name is involved in a circular definition."
    }
else {
    $wmidependents = (get-service $TargetService).dependentservices

    $wmidependentservices = Get-WmiObject Win32_Service | Select-object name,state,startmode | where {$wmidependents.name -contains $_.name}

    # Write-Host $TargetService

    foreach ($service in $wmidependentservices){
        if($service.startmode -eq "auto" -or $service.status -eq "Running"){
            # Write-Host "-> $($service.name)"
            CollectDependent($service.name)
        } 
        else{
            Write-Warning "Omitting $($service.name) : service is $($service.state) with the startmode: $($service.startmode)"
        }
    }

    stop-service $TargetService -ErrorAction SilentlyContinue

    $global:ServicesToStart += $TargetService
}

}

CollectDependent("SSTPSvc")

CollectDependent("IAS")

Write-Warning "Restarting services in the following order: " foreach($service in $global:ServicesToStart ) { Write-Warning $service }

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Warning "Done!"

latinkreationz commented 3 years ago

Serge,

Thank you so much!

latinkreationz commented 3 years ago

Dumb question as I'm still learning - do I just copy and paste the script into a ps1 file?

SergeCaron commented 3 years ago

Yep!

Save the script in some directory other the CTW directory.

Regards,

De : latinkreationz notifications@github.com Envoyé : 3 février 2021 23:24 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Dumb question as I'm still learning - do I just copy and paste the script into a ps1 file?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-773018023, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4BBW7S5GJAQ7GJSAITS5IOPTANCNFSM4PYY2OMQ.

latinkreationz commented 3 years ago

Great! Thanks again!!!

devast8tor commented 3 years ago

It doesn't look like either of Serge's posts formatted correctly in the forum thread... I took the liberty of cleaning up the code and applying the order and dependency logic changes per his revisions. Hope this helps!

##******************************************************************
## Revision date: 2020.12.20
##
## Copyright (c) 2020 PC-Évolution enr.
## This code is licensed under the GNU General Public License (GPL).
##
## THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF
## ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY
## IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR
## PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.
##
##******************************************************************

# Certify The Web Deployment Script for Windows Server 2016 Essentials (Your Mileage May Vary ;-)

# Restart the "Network Policy Server" as well as the SSTP protocol : all dependent services are
# stopped and started in the exact reverse order they were stopped.

# Note: the Write-Warning is used so that messages will appear in the Certify The Web log file.

# 2020.08.19: Initial version
# 2020.12.20: Change order of CollectDependant calls to addres some hidden dependencies.

param($result)

Import-Module RemoteDesktopServices

# Apply certificate
Set-Item -Path RDS:\GatewayServer\SSLCertificate\Thumbprint -Value  $result.ManagedItem.CertificateThumbprintHash -ErrorAction Stop

# Restart services required for "Access Anywhere"

$global:ServicesToStart = @()

Function CollectDependent($TargetService)

{
     # Caution: minimal effort done to prevent circular definitions
     if ($global:ServicesToStart -match "$service.name") {
                Write-Warning "Caution! Service $service.name is involved in a circular definition."
          }
     else {
          $wmidependents = (get-service $TargetService).dependentservices

          $wmidependentservices = Get-WmiObject Win32_Service | Select-object name,state,startmode | where {$wmidependents.name -contains $_.name}

          # Write-Host $TargetService

          foreach ($service in $wmidependentservices){
                if($service.startmode -eq "auto" -or $service.status -eq "Running"){
                     # Write-Host "-> $($service.name)"
                     CollectDependent($service.name)
                }
                else{
                     Write-Warning "Omitting $($service.name) : service is $($service.state) with the startmode: $($service.startmode)"
                }
          }

          stop-service $TargetService -ErrorAction SilentlyContinue

          $global:ServicesToStart += $TargetService
     }

}

CollectDependent("SSTPSvc")

CollectDependent("IAS")

Write-Warning "Restarting services in the following order: "
foreach($service in $global:ServicesToStart ) { Write-Warning $service }

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Warning "Done!"
latinkreationz commented 3 years ago

Thanks devast8tor!

devast8tor commented 3 years ago

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

SergeCaron commented 3 years ago

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 28 février 2021 23:14 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-787622784, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ.

devast8tor commented 3 years ago

Serge,

Appreciate your reply and I totally understand. I agree that there seems to be some undocumented shenanigans occurring in steps that are hidden behind the setup/repair wizard. I am happy to enable additional logging or to setup tracing (either of ctw or the wse setup wizard) to see if we can't tease out the voodoo occurring behind that setup progress bar.

If you'd like to collaborate, just let me know. I also have access to Unified Support via my employer, so have been considering opening a support case to capture this detail if it would help?

Regards, Ryan

On Mon, Mar 1, 2021, 4:27 AM SergeCaron notifications@github.com wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 28 février 2021 23:14 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author < author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-787911099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ .

latinkreationz commented 3 years ago

While the script works for me, the only issue I have is with SSTP VPN routing not working until a server reboot as Serge pointed out. So until a solution is mentioned, I went ahead and added a task within CTW to run a reboot after the script completes. It makes sense to have 3-5 minutes of down time vs. hours of not knowing it's down.

SergeCaron commented 3 years ago

Hello Ryan,

I like « shenanigans » : magic is always a nice escape also …

It is the architecture that I don’t fully grasp.

Once the certificate is renewed, the SSTP service will process incoming connections. The network stack will assign an IP and maintain the connection with keepalives (this is an oversimplification but the gist is that the VPN is established). In the snippet below, you can see that both users got exactly 343 bytes out of the server, regardless of connection duration.

The culprit seems to be NPS (the “Network Policy Server”) which does not seem to allow any incoming traffic on the connection.

In Install and Configure the NPS Server | Microsoft Docshttps://docs.microsoft.com/en-us/windows-server/remote/remote-access/vpn/always-on-vpn/deploy/vpn-deploy-nps, there is documentation concerning “Autoenroll the NPS Server Certificate” and a required Group Policy Update that I don’t fully understand. Just out of curiosity, I added a “invoke-gpudate” command in my script to see if this had any effect (and yes, the -force parameter is specified ;-).

At this point, this is the most plausible culprit. I just don’t understand why the issue is not the same on all servers.

So, if you want to further this … be my guest ;-)

Regards,

Serge Caron

PS C:\Windows\system32> Get-RemoteAccessConnectionStatistics | fl *

ClientIPv4Address : 172.16.0.70 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.70 AuthMethod : MSChapv2 ConnectionDuration : 2019 ConnectionStartTime : 2021-03-04 19:37:25 ConnectionType : Vpn HostName : - TotalBytesIn : 6683429 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 91 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user1} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

ClientIPv4Address : 172.16.0.72 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.72 AuthMethod : MSChapv2 ConnectionDuration : 4393 ConnectionStartTime : 2021-03-04 18:57:50 ConnectionType : Vpn HostName : - TotalBytesIn : 50809 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 12 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user2} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

PS C:\Windows\system32>

De : devast8tor notifications@github.com Envoyé : 3 mars 2021 11:34 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Appreciate your reply and I totally understand. I agree that there seems to be some undocumented shenanigans occurring in steps that are hidden behind the setup/repair wizard. I am happy to enable additional logging or to setup tracing (either of ctw or the wse setup wizard) to see if we can't tease out the voodoo occurring behind that setup progress bar.

If you'd like to collaborate, just let me know. I also have access to Unified Support via my employer, so have been considering opening a support case to capture this detail if it would help?

Regards, Ryan

On Mon, Mar 1, 2021, 4:27 AM SergeCaron notifications@github.com<mailto:notifications@github.com> wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor notifications@github.com<mailto:notifications@github.com> Envoyé : 28 février 2021 23:14 À : webprofusion/certify certify@noreply.github.com<mailto:certify@noreply.github.com> Cc : Serge Caron SCaron@pcevolution.com<mailto:SCaron@pcevolution.com>; Author < author@noreply.github.commailto:author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-787911099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-789852001, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4C3KS57YA3L3Y2XXWLTBZQGRANCNFSM4PYY2OMQ.

SergeCaron commented 3 years ago

Hello Ryan,

Sometimes, you need to read the documentation the way it is written. What needs to be done is to update the group policies the same way the update would happen on a reboot.

In the script, add the following line in the history:

2020.12.20 : Change order of CollectDependent calls to address some hidden dependencies.

2021.03.07 : Force NPS to update the shared secret that is the server certificate

Add the following lines after the services are stopped and before they are restarted (the first and last lines are duplicated here just to be certain ;-) : [array]::Reverse($global:ServicesToStart) <--- after this line

Write-Warning "Updating group policies"

The following command does not yield any output :

Invoke-GPUpdate -RandomDelayInMinutes 0 -Force

$GP = gpupdate /force foreach($line in $GP ) { Write-Warning $line } Write-Warning ""

Write-Warning "Restarting services in the following order: " <--- before this line

Save the updated script and proceed to a certificate renewal. You should be able to ping your domain default gateway from the VPN client ;-) (Actually, I waited about 10 seconds after the VPN connected to do that).

By the way, it so happens that on all other servers, the first VPN connection was more than two hours following a certificate renewal.

Kind regards,

Serge Caron

De : Serge Caron Envoyé : 4 mars 2021 23:25 À : 'webprofusion/certify' reply@reply.github.com; webprofusion/certify certify@noreply.github.com Cc : Author author@noreply.github.com Objet : RE: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hello Ryan,

I like « shenanigans » : magic is always a nice escape also …

It is the architecture that I don’t fully grasp.

Once the certificate is renewed, the SSTP service will process incoming connections. The network stack will assign an IP and maintain the connection with keepalives (this is an oversimplification but the gist is that the VPN is established). In the snippet below, you can see that both users got exactly 343 bytes out of the server, regardless of connection duration.

The culprit seems to be NPS (the “Network Policy Server”) which does not seem to allow any incoming traffic on the connection.

In Install and Configure the NPS Server | Microsoft Docshttps://docs.microsoft.com/en-us/windows-server/remote/remote-access/vpn/always-on-vpn/deploy/vpn-deploy-nps, there is documentation concerning “Autoenroll the NPS Server Certificate” and a required Group Policy Update that I don’t fully understand. Just out of curiosity, I added a “invoke-gpudate” command in my script to see if this had any effect (and yes, the -force parameter is specified ;-).

At this point, this is the most plausible culprit. I just don’t understand why the issue is not the same on all servers.

So, if you want to further this … be my guest ;-)

Regards,

Serge Caron

PS C:\Windows\system32> Get-RemoteAccessConnectionStatistics | fl *

ClientIPv4Address : 172.16.0.70 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.70 AuthMethod : MSChapv2 ConnectionDuration : 2019 ConnectionStartTime : 2021-03-04 19:37:25 ConnectionType : Vpn HostName : - TotalBytesIn : 6683429 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 91 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user1} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

ClientIPv4Address : 172.16.0.72 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.72 AuthMethod : MSChapv2 ConnectionDuration : 4393 ConnectionStartTime : 2021-03-04 18:57:50 ConnectionType : Vpn HostName : - TotalBytesIn : 50809 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 12 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user2} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

PS C:\Windows\system32>

De : devast8tor notifications@github.com Envoyé : 3 mars 2021 11:34 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Appreciate your reply and I totally understand. I agree that there seems to be some undocumented shenanigans occurring in steps that are hidden behind the setup/repair wizard. I am happy to enable additional logging or to setup tracing (either of ctw or the wse setup wizard) to see if we can't tease out the voodoo occurring behind that setup progress bar.

If you'd like to collaborate, just let me know. I also have access to Unified Support via my employer, so have been considering opening a support case to capture this detail if it would help?

Regards, Ryan

On Mon, Mar 1, 2021, 4:27 AM SergeCaron notifications@github.com<mailto:notifications@github.com> wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor notifications@github.com<mailto:notifications@github.com> Envoyé : 28 février 2021 23:14 À : webprofusion/certify certify@noreply.github.com<mailto:certify@noreply.github.com> Cc : Serge Caron SCaron@pcevolution.com<mailto:SCaron@pcevolution.com>; Author < author@noreply.github.commailto:author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-787911099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-789852001, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4C3KS57YA3L3Y2XXWLTBZQGRANCNFSM4PYY2OMQ.

latinkreationz commented 3 years ago

I added and tested the group policy update that you indicated and it didn't work for me. I checked the log and it ran successfully so not sure why it didn't work. Rebooting the server does make it work again so I guess I'll just continue using that extra task since it does work.

SergeCaron commented 3 years ago

Hello,

I am beginning to think the behavior on a “Windows Server 2016 Essentials” edition is not exactly the same as on a “Windows Server 2016 Standard” edition with the “Essentials Experience Role” installed.

I manage 4 servers of the first type and 3 of the second using this “Essentials” software.

I added some details concerning the renewed certificate and VPN configuration in the output log. Below is a sample log.

The revised code is:

[array]::Reverse($global:ServicesToStart) <--- after this line

Write-Warning "-----------------------" Write-Warning "Updating group policies"

The folloing command does not yield any output :

Invoke-GPUpdate -RandomDelayInMinutes 0 -Force

$GP = gpupdate /force $GP = [string]::Join([Environment]::NewLine, ($GP | ?{$_ -match ".."})) $GP | Write-Output Write-Warning ""

Write-Warning "Restarting services in the following order: " foreach($service in $global:ServicesToStart ) { Write-Warning $service } Write-Warning ""

foreach($service in $global:ServicesToStart ) { start-service $service -ErrorAction SilentlyContinue }

Write-Warning "Active certificate details: " $Status = Get-RemoteAccess | fl SSLCertificate, VPNStatus, VPNConfiguration | Out-String $Status = [string]::Join([Environment]::NewLine, ($Status | ?{$_ -match ".."})) $Status | Write-Output Write-Warning ""

Write-Warning "Done!" <--- before this line

On a “Windows Server 2016 Essentials”, OS version 14393.3930, I can assure you that the certificate is renewed and the VPN connections are normal hereafter. These are simple domains.

On a “Windows Server 2016 Standard” edition with the “Essentials Experience Role”, OS version 14393.4169, I can assure you that the VPN DOES NOT PASS TRAFIC, although the certificate is renewed and the VPN connections are normal hereafter. This is a more complex domain with 2 domain controllers but this should not be a variable in this issue.

Both tests are run from the same physical client and from the same user account: obviously, this should not have an effect on the remote server but I just wanted to be sure that the client did not interfere.

I will let the second domain alone for two hours in case the domain GPO refresh is significant, but I believe the issue is elsewhere.

Regards,

Serge Caron

Sample log output:

2021-03-08 15:57:54.071 -05:00 [INF] Executing command via PowerShell 2021-03-08 15:58:27.527 -05:00 [INF] Omitting RasMan : service is Stopped with the startmode: Disabled Omitting rqs : service is Stopped with the startmode: Manual Attente de l’arrêt du service « Routage et accès distant (RemoteAccess) »… Attente de l’arrêt du service « Routage et accès distant (RemoteAccess) »… Omitting rqs : service is Stopped with the startmode: Manual Attente de l’arrêt du service « Network Policy Server (IAS) »… Attente de l’arrêt du service « Network Policy Server (IAS) »…

Updating group policies Mise … jour de la strat‚gie... La mise … jour de la strat‚gie d'ordinateur s'est termin‚e sans erreur. La mise … jour de la strat‚gie utilisateur s'est termin‚e sans erreur.

Restarting services in the following order: IAS TSGateway SSTPSvc RemoteAccess

Active certificate details:

SslCertificate : [Subject] CN=passerelle.

               [Issuer]
                 CN=R3, O=Let's Encrypt, C=US

               [Serial Number]
                 03A8059FBxxxxxxF6A41528D6926E6058CDE

               [Not Before]
                 2021-03-08 14:57:50

               [Not After]
                 2021-06-06 15:57:50

               [Thumbprint]
                 684004887xxxxxx8F639D58CC8D3F387F594911C

VPNStatus : Installed VPNConfiguration : VirtualPrivateNetworkConfiguration

Done!

2021-03-08 15:58:27.527 -05:00 [INF] Run Powershell Script :: Task Completed OK 2021-03-08 15:58:27.534 -05:00 [INF] Request completed

De : latinkreationz notifications@github.com Envoyé : 7 mars 2021 23:53 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

I added and tested the group policy update that you indicated and it didn't work for me. I checked the log and it ran successfully so not sure why it didn't work. Rebooting the server does make it work again so I guess I'll just continue using that extra task since it does work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-792460737, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4ECZJSGZRLCU2NW75LTCRJ2XANCNFSM4PYY2OMQ.

latinkreationz commented 3 years ago

Very interesting. I'm actually running Windows Server 2016 Essentials version 14393.3930 and the SSTP VPN was still not routing after a successful renewal. Just to confirm - is the revised code specific to Server 2016 Standard or should it work regardless? I can continue to do some further testing on my end as well. I'm not as technical as you guys are but I'll do what I can. Let me know.

SergeCaron commented 3 years ago

Hello,

Well, you are running the same edition and version as the server for which the revised code is working at my end.

Quick tip: you can update the code and rerun the deployment task without requesting a new certificate. This way, you can test an external client against you server.

I am currently updating the test server to the latest (and greatest!) Windows Server 2016 patches. I still expect this will work tomorrow morning.

Regards.

Serge Caron

De : latinkreationz notifications@github.com Envoyé : 8 mars 2021 19:24 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Very interesting. I'm actually running Windows Server 2016 Essentials version 14393.3930 and the SSTP VPN was still not routing after a successful renewal. Just to confirm - is the revised code specific to Server 2016 Standard or should it work regardless? I can continue to do some further testing on my end as well. I'm not as technical as you guys are but I'll do what I can. Let me know.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-793196629, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4DYJMFELJD7CVIQEUDTCVTA5ANCNFSM4PYY2OMQ.

webprofusion-chrisc commented 3 years ago

I'd recommend raising a ticket with Microsoft for specific instructions regarding your version of Windows server. Clearly you've got your certificate OK and the problem (which is specific to Microsoft services) is how to apply it cleanly. I'd wager that they'd will tell you to apply the certificate then restart the server.

Note again that Deployment Tasks can be setup to run Manually, this lets you have certificate be auto-renewed by Certify but the actual deployment can be a manual task during a maintenance window (e.g. monthly patching).

SergeCaron commented 3 years ago

Hello Christopher,

Thank you for the suggestion.

That is where I am actually heading. I would like to avoid the manual reboot if at all possible but if everything else fails …

I would also like to have a clearer understanding of the “improved” RRAS at the SSTP level: I have a number of technical staff travelling the world and HTTPS is often the only protocol allowed out of hotels and other public places. Clearly, SSTPSvc is disjoint from NPC (IAS) at the surface level but seems to be connected through some other mechanism.

Kind regards,

Serge Caron

De : Christopher Cook notifications@github.com Envoyé : 8 mars 2021 20:12 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

I'd recommend raising a ticket with Microsoft for specific instructions regarding your version of Windows server. Clearly you've got your certificate OK and the problem (which is specific to Microsoft services) is how to apply it cleanly. I'd wager that they'd will tell you to apply the certificate then restart the server.

Note again that Deployment Tasks can be setup to run Manually, this lets you have certificate be auto-renewed by Certify but the actual deployment can be a manual task during a maintenance window (e.g. monthly patching).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-793231114, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4ETESUPKOBREQK2SX3TCVYVZANCNFSM4PYY2OMQ.

webprofusion-chrisc commented 3 years ago

Yes I think to fully solve this issue you'd need specialist advice from the part of the Windows product team at Microsoft who manage this feature. It's entirely possible that the process cannot be cleanly automated in all versions of windows server, because around 2016 it was normal for certificates to only be replaced every year or two.

devast8tor commented 3 years ago

Serge,

FWIW, I'm running Windows Server 2016 Essentials (v1607, OS Version 14393.4225). There must also be some differences between the Standard version with the role and the Essentials version because using your script always results in an error being tripped for Terminal Service Gateway using the old cert.

On Mon, Mar 1, 2021 at 4:27 AM SergeCaron notifications@github.com wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 28 février 2021 23:14 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author < author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-787911099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ .

devast8tor commented 3 years ago

Serge,

Thank you for supplying the GP update lines for the script. I just wanted to point out I might still be using an older version of the script as my code does not have the following line:

[array]::Reverse($global:ServicesToStart) <--- after this line

Logically, I inserted this after the CollectDependent("IAS"), but before the service restart section. Is there a newer version of the script where you've rewritten the service dependencies collection and stop logic?

On Sun, Mar 7, 2021 at 6:31 AM SergeCaron notifications@github.com wrote:

Hello Ryan,

Sometimes, you need to read the documentation the way it is written. What needs to be done is to update the group policies the same way the update would happen on a reboot.

In the script, add the following line in the history:

2020.12.20 : Change order of CollectDependent calls to address some

hidden dependencies.

2021.03.07 : Force NPS to update the shared secret that is the server

certificate

Add the following lines after the services are stopped and before they are restarted (the first and last lines are duplicated here just to be certain ;-) : [array]::Reverse($global:ServicesToStart) <--- after this line

Write-Warning "Updating group policies"

The following command does not yield any output :

Invoke-GPUpdate -RandomDelayInMinutes 0 -Force

$GP = gpupdate /force foreach($line in $GP ) { Write-Warning $line } Write-Warning ""

Write-Warning "Restarting services in the following order: " <--- before this line

Save the updated script and proceed to a certificate renewal. You should be able to ping your domain default gateway from the VPN client ;-) (Actually, I waited about 10 seconds after the VPN connected to do that).

By the way, it so happens that on all other servers, the first VPN connection was more than two hours following a certificate renewal.

Kind regards,

Serge Caron

De : Serge Caron Envoyé : 4 mars 2021 23:25 À : 'webprofusion/certify' reply@reply.github.com; webprofusion/certify certify@noreply.github.com Cc : Author author@noreply.github.com Objet : RE: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hello Ryan,

I like « shenanigans » : magic is always a nice escape also …

It is the architecture that I don’t fully grasp.

Once the certificate is renewed, the SSTP service will process incoming connections. The network stack will assign an IP and maintain the connection with keepalives (this is an oversimplification but the gist is that the VPN is established). In the snippet below, you can see that both users got exactly 343 bytes out of the server, regardless of connection duration.

The culprit seems to be NPS (the “Network Policy Server”) which does not seem to allow any incoming traffic on the connection.

In Install and Configure the NPS Server | Microsoft Docs< https://docs.microsoft.com/en-us/windows-server/remote/remote-access/vpn/always-on-vpn/deploy/vpn-deploy-nps>, there is documentation concerning “Autoenroll the NPS Server Certificate” and a required Group Policy Update that I don’t fully understand. Just out of curiosity, I added a “invoke-gpudate” command in my script to see if this had any effect (and yes, the -force parameter is specified ;-).

At this point, this is the most plausible culprit. I just don’t understand why the issue is not the same on all servers.

So, if you want to further this … be my guest ;-)

Regards,

Serge Caron

PS C:\Windows\system32> Get-RemoteAccessConnectionStatistics | fl *

ClientIPv4Address : 172.16.0.70 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.70 AuthMethod : MSChapv2 ConnectionDuration : 2019 ConnectionStartTime : 2021-03-04 19:37:25 ConnectionType : Vpn HostName : - TotalBytesIn : 6683429 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 91 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user1} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

ClientIPv4Address : 172.16.0.72 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.72 AuthMethod : MSChapv2 ConnectionDuration : 4393 ConnectionStartTime : 2021-03-04 18:57:50 ConnectionType : Vpn HostName : - TotalBytesIn : 50809 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 12 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user2} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

PS C:\Windows\system32>

De : devast8tor notifications@github.com Envoyé : 3 mars 2021 11:34 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author < author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Appreciate your reply and I totally understand. I agree that there seems to be some undocumented shenanigans occurring in steps that are hidden behind the setup/repair wizard. I am happy to enable additional logging or to setup tracing (either of ctw or the wse setup wizard) to see if we can't tease out the voodoo occurring behind that setup progress bar.

If you'd like to collaborate, just let me know. I also have access to Unified Support via my employer, so have been considering opening a support case to capture this detail if it would help?

Regards, Ryan

On Mon, Mar 1, 2021, 4:27 AM SergeCaron <notifications@github.com<mailto: notifications@github.com>> wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor <notifications@github.com<mailto: notifications@github.com>> Envoyé : 28 février 2021 23:14 À : webprofusion/certify <certify@noreply.github.com<mailto: certify@noreply.github.com>> Cc : Serge Caron SCaron@pcevolution.com<mailto:SCaron@pcevolution.com>; Author < author@noreply.github.commailto:author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<

https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>,

or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/webprofusion/certify/issues/519#issuecomment-787911099>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ>

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-789852001>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4C3KS57YA3L3Y2XXWLTBZQGRANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-792288406, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGOQZOV5Q3R2XJKR5DTCOE43ANCNFSM4PYY2OMQ .

SergeCaron commented 3 years ago

Hello Ryan, Christopher ad all ;-)

First, there is a missing piece in the puzzle. In the “Essentials” internal architecture, there seems to be some dependencies on IPv6 and Configure IPv6 for advanced users - Windows Server | Microsoft Docshttps://docs.microsoft.com/en-US/troubleshoot/windows-server/networking/configure-ipv6-in-windows recommends leaving it on.

There is a service, IPHelpSvc, that translates protocols between IPv4 and IPv6, including IP-HTTPS. Which means that this service depends on the certificate being replaced, whatever the way it gets it.

So I decided to add this service (and dependents) to the list and updated the script.

Second, we have this issue across Windows Server 2016 editions. It is not so obvious:

So, I am going to upgrade the recalcitrant server just to see if this issue is OS version related. Basically, I know that .4169 is only a month older than .4225 but I would like to have identical server versions that yield different result for the same configuration.

In the meantime, you can ask Christopher to unlock the attached file ;-)

Kind regards,

Serge Caron

De : devast8tor notifications@github.com Envoyé : 9 mars 2021 00:40 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author author@noreply.github.com Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Thank you for supplying the GP update lines for the script. I just wanted to point out I might still be using an older version of the script as my code does not have the following line:

[array]::Reverse($global:ServicesToStart) <--- after this line

Logically, I inserted this after the CollectDependent("IAS"), but before the service restart section. Is there a newer version of the script where you've rewritten the service dependencies collection and stop logic?

On Sun, Mar 7, 2021 at 6:31 AM SergeCaron notifications@github.com wrote:

Hello Ryan,

Sometimes, you need to read the documentation the way it is written. What needs to be done is to update the group policies the same way the update would happen on a reboot.

In the script, add the following line in the history:

2020.12.20 : Change order of CollectDependent calls to address some

hidden dependencies.

2021.03.07 : Force NPS to update the shared secret that is the server

certificate

Add the following lines after the services are stopped and before they are restarted (the first and last lines are duplicated here just to be certain ;-) : [array]::Reverse($global:ServicesToStart) <--- after this line

Write-Warning "Updating group policies"

The following command does not yield any output :

Invoke-GPUpdate -RandomDelayInMinutes 0 -Force

$GP = gpupdate /force foreach($line in $GP ) { Write-Warning $line } Write-Warning ""

Write-Warning "Restarting services in the following order: " <--- before this line

Save the updated script and proceed to a certificate renewal. You should be able to ping your domain default gateway from the VPN client ;-) (Actually, I waited about 10 seconds after the VPN connected to do that).

By the way, it so happens that on all other servers, the first VPN connection was more than two hours following a certificate renewal.

Kind regards,

Serge Caron

De : Serge Caron Envoyé : 4 mars 2021 23:25 À : 'webprofusion/certify' reply@reply.github.com; webprofusion/certify certify@noreply.github.com Cc : Author author@noreply.github.com Objet : RE: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hello Ryan,

I like « shenanigans » : magic is always a nice escape also …

It is the architecture that I don’t fully grasp.

Once the certificate is renewed, the SSTP service will process incoming connections. The network stack will assign an IP and maintain the connection with keepalives (this is an oversimplification but the gist is that the VPN is established). In the snippet below, you can see that both users got exactly 343 bytes out of the server, regardless of connection duration.

The culprit seems to be NPS (the “Network Policy Server”) which does not seem to allow any incoming traffic on the connection.

In Install and Configure the NPS Server | Microsoft Docs< https://docs.microsoft.com/en-us/windows-server/remote/remote-access/vpn/always-on-vpn/deploy/vpn-deploy-nps>, there is documentation concerning “Autoenroll the NPS Server Certificate” and a required Group Policy Update that I don’t fully understand. Just out of curiosity, I added a “invoke-gpudate” command in my script to see if this had any effect (and yes, the -force parameter is specified ;-).

At this point, this is the most plausible culprit. I just don’t understand why the issue is not the same on all servers.

So, if you want to further this … be my guest ;-)

Regards,

Serge Caron

PS C:\Windows\system32> Get-RemoteAccessConnectionStatistics | fl *

ClientIPv4Address : 172.16.0.70 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.70 AuthMethod : MSChapv2 ConnectionDuration : 2019 ConnectionStartTime : 2021-03-04 19:37:25 ConnectionType : Vpn HostName : - TotalBytesIn : 6683429 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 91 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user1} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

ClientIPv4Address : 172.16.0.72 ClientIPv6Address : :: ClientExternalAddress : [ snipped ] ClientIPAddress : 172.16.0.72 AuthMethod : MSChapv2 ConnectionDuration : 4393 ConnectionStartTime : 2021-03-04 18:57:50 ConnectionType : Vpn HostName : - TotalBytesIn : 50809 TotalBytesOut : 343 TransitionTechnology : None TunnelType : Sstp Bandwidth : 12 RoutingDomain : UserActivityState : Active UserName : {DOMAIN\user2} PSComputerName : CimClass : root/Microsoft/Windows/RemoteAccess : RemoteAccessMonitoringConnection CimInstanceProperties : {AuthMethod, ClientExternalAddress, ClientIPv4Address, ClientIPv6Address...} CimSystemProperties : Microsoft.Management.Infrastructure.CimSystemProperties

PS C:\Windows\system32>

De : devast8tor notifications@github.com Envoyé : 3 mars 2021 11:34 À : webprofusion/certify certify@noreply.github.com Cc : Serge Caron SCaron@pcevolution.com; Author < author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Serge,

Appreciate your reply and I totally understand. I agree that there seems to be some undocumented shenanigans occurring in steps that are hidden behind the setup/repair wizard. I am happy to enable additional logging or to setup tracing (either of ctw or the wse setup wizard) to see if we can't tease out the voodoo occurring behind that setup progress bar.

If you'd like to collaborate, just let me know. I also have access to Unified Support via my employer, so have been considering opening a support case to capture this detail if it would help?

Regards, Ryan

On Mon, Mar 1, 2021, 4:27 AM SergeCaron <notifications@github.com<mailto: notifications@github.com>> wrote:

Hello Ryan,

That’s interesting.

We are running Windows Server 2016 Standard (v1607, OS Version 14393.4169): I have 7 different instances running and serving roughly 200 users.

These instances are using only SSTP to access the VPN. On one of these instances we had a month ago a situation where the Terminal Services Gateway was working perfectly following the certificate renewal but not the SSTP service which uses the same certificate.

We believe that there is some undocumented relationship between all these services: in the case above, everything was back to normal following a reboot. Manually stopping all the KNOWN services are restarting in the correct order was no help: reboot was the only solution that worked in this case.

The issue is on my TODO list.

I can’t report any error on the six other servers.

Regards,

Serge Caron

De : devast8tor <notifications@github.com<mailto: notifications@github.com>> Envoyé : 28 février 2021 23:14 À : webprofusion/certify <certify@noreply.github.com<mailto: certify@noreply.github.com>> Cc : Serge Caron SCaron@pcevolution.com<mailto:SCaron@pcevolution.com>; Author < author@noreply.github.commailto:author@noreply.github.com> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Hey Serge,

Just wanted to let you know that the updated script still isn't working right for me, even though the log didn't show any errors. After my last cert update on 2/24, the remote access server started throwing cert errors again. I ended up applying the same manual fix I've used before, which was to export the new cert from IIS and then re-rerun WSE remote setup through the wizard.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<

https://github.com/webprofusion/certify/issues/519#issuecomment-787622784>,

or unsubscribe<

https://github.com/notifications/unsubscribe-auth/AQROU4CCTXBSXLVYO6C6V53TBMH73ANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/webprofusion/certify/issues/519#issuecomment-787911099>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AIJWUSGMJS3KGAG5COXOV7DTBOB2ZANCNFSM4PYY2OMQ>

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub< https://github.com/webprofusion/certify/issues/519#issuecomment-789852001>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AQROU4C3KS57YA3L3Y2XXWLTBZQGRANCNFSM4PYY2OMQ>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/webprofusion/certify/issues/519#issuecomment-792288406, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJWUSGOQZOV5Q3R2XJKR5DTCOE43ANCNFSM4PYY2OMQ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-793430010, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4FAODZDJQZOWAHMTA3TCWYETANCNFSM4PYY2OMQ.

webprofusion-chrisc commented 3 years ago

@SergeCaron thanks for your investigation! To attach files you need to replying using the github user interface, I don't get your attachment either and there is no option for me to unlock it etc.

SergeCaron commented 3 years ago

Hello Christopher,

I finally broke down and got myself a network trace of a SSTP VPN client connection both before and after a certificate renewal.

In the samples below, the remote client is pinging the Google DNS 8.8.8.8.

Both before and after the certificate renewal, we can see the Client/Server key exchange (packets 4 to 9 and 181 to 187).

Now, each ICMP request from the client is a TCP "conversation". Before the certificate renewal, you can see a full Echo/Reply over packets 148 to 152 and then 153 to 157. These are conversations #15688 and #15837.

After the certificate is renewed, you can see that the network stack does not RETURN the information to the VPN client.

In packets 388 to 390, you can see the ping request coming from the VPN client, going over the destination and returning to the server. However, the resulting frame is not going to the VPN client: this and all subsequent packets are all part of the SAME conversation #5127.

The network stack does not seem to be able to handle the certificate replacement even if it properly processed the handshaking using the new cetificate.

A reboot appears to simply restart the network stack in healthy conditions.

We have this issue on two servers out of 8. I do not know what the discriminaton factor is.

What I do know is that users should be aware of this issue.

Regards,

Serge Caron

No. Time Source Destination Protocol Length Info 4 0.016066 [ VPNClient ] 192.168.33.4 TLSv1.2 257 Client Hello 5 0.020978 192.168.33.4 [ VPNClient ] TCP 1514 443 → 21120 [ACK] Seq=1 Ack=204 Win=65536 Len=1460 [TCP segment of a reassembled PDU] 6 0.020980 192.168.33.4 [ VPNClient ] TCP 1514 443 → 21120 [ACK] Seq=1461 Ack=204 Win=65536 Len=1460 [TCP segment of a reassembled PDU] 7 0.020981 192.168.33.4 [ VPNClient ] TLSv1.2 569 Server Hello, Certificate, Certificate Status, Server Key Exchange, Server Hello Done 8 0.037818 [ VPNClient ] 192.168.33.4 TCP 56 21120 → 443 [ACK] Seq=204 Ack=2921 Win=65536 Len=0 9 0.045986 [ VPNClient ] 192.168.33.4 TLSv1.2 236 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message

148 39.907930 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 149 39.908281 192.168.33.67 [ 8.8.8.8 ] ICMP 74 Echo (ping) request id=0x0001, seq=32/8192, ttl=127 (reply in 150) 150 39.911243 [ 8.8.8.8 ] 192.168.33.67 ICMP 74 Echo (ping) reply id=0x0001, seq=32/8192, ttl=116 (request in 149) 151 39.911469 192.168.33.4 [ VPNClient ] TLSv1.2 203 Application Data 152 39.975767 [ VPNClient ] 192.168.33.4 TCP 56 21120 → 443 [ACK] Seq=15688 Ack=9892 Win=65280 Len=0

153 40.919939 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 154 40.920178 192.168.33.67 [ 8.8.8.8 ] ICMP 74 Echo (ping) request id=0x0001, seq=33/8448, ttl=127 (reply in 155) 155 40.923065 [ 8.8.8.8 ] 192.168.33.67 ICMP 74 Echo (ping) reply id=0x0001, seq=33/8448, ttl=116 (request in 154) 156 40.923232 192.168.33.4 [ VPNClient ] TLSv1.2 203 Application Data 157 40.995881 [ VPNClient ] 192.168.33.4 TCP 56 21120 → 443 [ACK] Seq=15837 Ack=10041 Win=65024 Len=0

No. Time Source Destination Protocol Length Info 181 263.820006 [ VPNClient ] 192.168.33.4 TLSv1.2 257 Client Hello 182 263.827433 192.168.33.4 [ VPNClient ] TCP 1514 443 → 63761 [ACK] Seq=1 Ack=204 Win=65536 Len=1460 [TCP segment of a reassembled PDU] 183 263.827444 192.168.33.4 [ VPNClient ] TCP 1514 443 → 63761 [ACK] Seq=1461 Ack=204 Win=65536 Len=1460 [TCP segment of a reassembled PDU] 184 263.827445 192.168.33.4 [ VPNClient ] TLSv1.2 568 Server Hello, Certificate, Certificate Status, Server Key Exchange, Server Hello Done 185 263.843711 [ VPNClient ] 192.168.33.4 TCP 56 63761 → 443 [ACK] Seq=204 Ack=3435 Win=65536 Len=0 186 263.849855 [ VPNClient ] 192.168.33.4 TLSv1.2 236 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message 187 263.851457 192.168.33.4 [ VPNClient ] TLSv1.2 161 Change Cipher Spec, Encrypted Handshake Message

388 326.284106 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 389 326.284400 192.168.33.111 [ 8.8.8.8 ] ICMP 74 Echo (ping) request id=0x0001, seq=44/11264, ttl=127 (reply in 390) 390 326.287364 [ 8.8.8.8 ] 192.168.33.111 ICMP 74 Echo (ping) reply id=0x0001, seq=44/11264, ttl=116 (request in 389) 391 326.343307 192.168.33.4 [ VPNClient ] TCP 54 443 → 63761 [ACK] Seq=5127 Ack=19864 Win=65024 Len=0

392 327.275837 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 393 327.327456 192.168.33.4 [ VPNClient ] TCP 54 443 → 63761 [ACK] Seq=5127 Ack=20013 Win=65024 Len=0 394 329.435889 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 395 329.487968 192.168.33.4 [ VPNClient ] TCP 54 443 → 63761 [ACK] Seq=5127 Ack=20162 Win=64768 Len=0

396 331.148776 [ VPNClient ] 192.168.33.4 TLSv1.2 203 Application Data 397 331.149130 192.168.33.111 [ 8.8.8.8 ] ICMP 74 Echo (ping) request id=0x0001, seq=45/11520, ttl=127 (reply in 398) 398 331.152060 [ 8.8.8.8 ] 192.168.33.111 ICMP 74 Echo (ping) reply id=0x0001, seq=45/11520, ttl=116 (request in 397) 399 331.206642 192.168.33.4 [ VPNClient ] TCP 54 443 → 63761 [ACK] Seq=5127 Ack=20311 Win=64768 Len=0

webprofusion-chrisc commented 3 years ago

Thanks for the detailed analysis @SergeCaron I really do think you should raise this directly with Microsoft for a support escalation, linking them to this github issue. Rebooting the server isn't really a solution and only Microsoft can really investigate this in detail.

Technically the issue is not with Certify because you already have your certificate. Although this is an interesting topic, the point at which we stop providing direct support (with the exception of IIS website bindings) is when you have the certificate then want to use it with another service (there are of course thousands of ways to use a certificate and they all have their own interesting problems). We try to provide the basic mechanism for you to apply certificates automatically but we don't directly support issues with individual services (because we don't have domain knowledge for every service).

SergeCaron commented 3 years ago

Hello Christopher,

You are, of course, entirely correct.

The documentation and support material is provided to make sure there is a valid certificate renewal and that this certificate is installed correctly and in use.

The fact that SSTP goes haywire has nothing to do with the certificate renewal, be it manually with some other supplier or using Certify The Web.

However, if the clients expect SSTP to work, then they MAY need to reboot their server. As they say, “Your mileage may vary.” ;-)

I will see if I can open a ticket with Microsoft: out of 8 servers, the only two for which I have this issue are production servers with very little maintenance windows.

Kind regards,

Serge Caron

De : Christopher Cook @.> Envoyé : 16 mars 2021 22:26 À : webprofusion/certify @.> Cc : Serge Caron @.>; Mention @.> Objet : Re: [webprofusion/certify] Windows Server Essentials 2016 Access Anywhere Certificate renewal script (#519)

Thanks for the detailed analysis @SergeCaronhttps://github.com/SergeCaron I really do think you should raise this directly with Microsoft for a support escalation, linking them to this github issue. Rebooting the server isn't really a solution and only Microsoft can really investigate this in detail.

Technically the issue is not with Certify because you already have your certificate. Although this is an interesting topic, the point at which we stop providing direct support (with the exception of IIS website bindings) is when you have the certificate then want to use it with another service (there are of course thousands of ways to use a certificate and they all have their own interesting problems). We try to provide the basic mechanism for you to apply certificates automatically but we don't directly support issues with individual services (because we don't have domain knowledge for every service).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/webprofusion/certify/issues/519#issuecomment-800743509, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQROU4D3NYZQLR3BRWO7BJDTEAHMZANCNFSM4PYY2OMQ.