search

Esri / arcgis-powershell-dsc

This repository contains scripts, code and samples for automating the install and configuration of ArcGIS (Enterprise and Desktop) using Microsoft Windows PowerShell DSC (Desired State Configuration).
Apache License 2.0
115 stars 62 forks source link

ArcGIS_License "The SendConfigurationApply function did not succeed" 10.8.1 - 10.9.1 upgrade #421

Open ericandersondhshq opened 2 years ago

ericandersondhshq commented 2 years ago

Community Note

Module Version

Affected Resource(s)

Configuration Files

{
    "AllNodes": [
        {
            "NodeName": "<RETRACTED>",
            "DataStoreTypes": [
                "Relational"
            ],
            "Role": [
                "DataStore"
            ]
        },
        {
            "NodeName": "<RETRACTED>",
            "Role": [
                "Portal"
            ]
        },
        {
            "NodeName": "<RETRACTED>",
            "Role": [
                "Server"
            ]
        },
        {
            "NodeName": "<RETRACTED>",
            "Role": [
                "PortalWebAdaptor",
                "ServerWebAdaptor"
            ]
        },
        {
            "NodeName": "<RETRACTED>",
            "Role": [
                "PortalWebAdaptor",
                "ServerWebAdaptor"
            ]
        }
    ],
    "ConfigData": {
        "Version": "10.9.1",
        "OldVersion": "10.8.1",
        "ServerContext": "host",
        "PortalContext": "gii",
        "ServerRole": "GeneralPurposeServer",
        "Credentials": {
            "ServiceAccount": {
                "Password": "<RETRACTED>",
                "UserName": "<RETRACTED>",
                "IsDomainAccount": true,
                "IsMSAAccount": false
            }
        },
        "Server": {
            "LicenseFilePath": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\licenses\\ArcGISGISServerAdvanced_ArcGISServer_1201091.prvc",
            "Installer": {
                "Path": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\ArcGIS_Server_Windows_1091_180041.exe",
                "InstallDir": "E:\\Programs\\ArcGIS\\Server",
                "InstallDirPython": "E:\\Python27",
                "EnableArcMapRuntime": true,
                "EnableDotnetSupport": false
            },
            "ServerDirectoriesRootLocation": "\\\\giishare\\prod\\arcgis\\hst\\arcgisserver\\directories",
            "ConfigStoreLocation": "\\\\giishare\\prod\\arcgis\\hst\\arcgisserver\\config-store",
            "PrimarySiteAdmin": {
                "UserName": "<RETRACTED>",
                "Password": "<RETRACTED>"
            }
        },
        "Portal": {
            "LicenseFilePath": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\licenses\\ArcGIS_Enterprise_Portal_1091_390116_20220727.json",
            "PortalLicenseUserTypeId": "creatorUT",
            "Installer": {
                "Path": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\Portal_for_ArcGIS_Windows_1091_180052.exe",
                "PatchesDir": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\patches\\portal",
                "WebStylesPath": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\Portal_for_ArcGIS_Web_Styles_Windows_1091_180053.exe",
                "InstallDir": "E:\\Programs\\ArcGIS\\Portal",
                "ContentDir": "E:\\arcgisportal"
            },
            "ContentDirectoryLocation": "E:\\arcgisportal\\content",
            "PortalAdministrator": {
                "UserName": "<RETRACTED>",
                "Email": "<RETRACTED>",
                "Password": "<RETRACTED>",
                "SecurityQuestionIndex": 1,
                "SecurityAnswer": "vanilla"
            }
        },
        "DataStore": {
            "ContentDirectoryLocation": "E:\\arcgisdatastore",
            "EnableFailoverOnPrimaryStop": false,
            "Installer": {
                "Path": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\ArcGIS_DataStore_Windows_1091_180054.exe",
                "PatchesDir": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\patches\\datastore",
                "InstallDir": "E:\\Programs\\ArcGIS\\DataStore"
            }
        },
        "WebAdaptor": {
            "AdminAccessEnabled": true,
            "Installer": {
                "Path": "\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\ArcGIS_Web_Adaptor_for_Microsoft_IIS_1091_180055.exe"
            }
        }
    }
}

Expected Behavior

Upgrade all ArcGIS Enterprise components from 10.8.1 to 10.9.1

Actual Behavior

When running the above config, it was able to upgrade the Portal for ArcGIS from 10.8.1 to 10.9.1 successfully. It is also able to upgrade the Portal web adaptors successfully. It looks like it successfully upgraded the ArcGIS Server as well, based on the version file within the config-store and what I see in :6443/arcgis/admin, however, the PowerShell script continues to fail with the below error.

9/26/2022 8:04:13 AM: PowerShell DSC resource ArcGIS_License failed to execute Set-TargetResource functionality with error message:
9/26/2022 8:04:14 AM: The SendConfigurationApply function did not succeed.

I am able to successfully license the product if I manually use the ArcGIS Enterprise software authorization .exe on the ArcGIS Server machine, which tells me there's no issue with the license file itself.

I am running the PowerShell command as an Admin, and I also tried passing in admin creds into the Invoke command, but it made no difference.

The service account has full permissions to the E:\arcgisserver and E:\Programs\ArcGIS\Server directories, as well as the config-store and directories which are on a file share.

Steps to Reproduce

Invoke-ArcGISConfiguration -ConfigurationParametersFile "\\giishare\prod\gis\powershell\gii\config\331\1091\green-prt-hst-dst.json" -Mode Upgrade -Credential <redacted> -DebugSwitch

I can run this multiple times, and it skips over the Portal/Portal web adaptors, since already upgraded, but always fails on the ArcGIS Server even though it looks to be successfully upgraded and I was able to license the software manually.

ServerUpgrade-9-25-2022-2-19-22-PM-Verbose.txt ServerUpgrade-9-25-2022-2-19-22-PM-Error.txt

ericandersondhshq commented 2 years ago

Hi @cameronkroeker - did you have any insight hopefully? Thanks in advance

rlhadsel commented 2 years ago

Hey @ericandersondhshq, have you tried licensing the ArcGIS Server software silently (similar to what the DSC is doing): https://enterprise.arcgis.com/en/server/10.9.1/install/windows/authorize-arcgis-server.htm#ESRI_SECTION1_2EF08A7261D248F596B749D3ED825460. This might help isolate any issues. I've also seen this step fail before due to the .prvc file not having all of the sections filled out/invalid characters for things like Organization, Department, etc. Can you confirm that these are all filled out and that the values are valid? Lastly, I've had luck generating a .ecp license file versus using a .prvc file in the past. Maybe generating this type of file and using it instead will help?

ericandersondhshq commented 2 years ago

Hi @rlhadsel - very much appreciate the response.

It appears like I am able to authorize the software silently with the .prvc file fine using the command line. It returns no errors when I do this. I'm also able to authorize the software fine by directly using the Software Authorization exe.

I can confirm there are no special characters within the user info of the prvc file. Only alphanumeric except for the obvious @ in email and hyphen in phone number.

We can certainly try with a .ecp file, but we've always used prvc in the past and have not had an issue with them in other environments upgrading to 10.9.1 and using same version of the ArcGIS Powershell DSC (3.3.1).

ericandersondhshq commented 2 years ago

Hi @rlhadsel , did you have any other suggestions hopefully?

rlhadsel commented 2 years ago

Hey @ericandersondhshq, thanks for checking the silent authorization outside of the PowerShell DSC Module. That is the tool that is used under the hood of the DSC, so it's curious that the DSC is failing. Have you tried the .ecp file? Also, you mentioned using PowerShell DSC version 3.3.1 (which should work just fine), but have you considered upgrading this to see if that resolves the issue?

H47110815G commented 2 years ago

In the last environment we built in December, online authorization with DSC 3.3.1 still worked.

Online authorization no longer works with *.prvc. We tested DSC 4.0.0 and DSC 3.3.2 - I didn't want to go any deeper because we want to use AG_Enterprise 11.

If you start the authorization via the license tool, you can get through it, so it's not the ESRI tool.

In the end we used the *.ecp. Unfortunately it's an extra step. Think it has to do with the security configuration of the systems again: WinServer2019.

ericandersondhshq commented 2 years ago

I can't find anything online about how to generate a .ecp license file, and I talked to our Esri License Admin and he doesn't know either, that option is not available from MyEsri. Can you expand on how to generate this type of license?

H47110815G commented 2 years ago

use the server's licensing assistant for the server and select offline licensing. You will then receive a config that you can save locally and then send to the email address specified in the assistant.

Have a nice weekend

ericandersondhshq commented 2 years ago

Is there any online documentation I can reference? I just ran through the "Software Authorization" tool on our ArcGIS Server machine and did not see the option for offline license like you are suggesting. Thank you for your help.

cameronkroeker commented 2 years ago

Hi @ericandersondhshq,

I suspect the issue is with the location of where the .prvc file is located. When using file shares in DSC, the NT Authority\Local System account of the target node will need access to the share. Looks like the installers (setup.exe) are also located in the same share but different folder. Here are some recommendations:

\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\licenses\\ArcGISGISServerAdvanced_ArcGISServer_1201091.prvc

For more information on how to grant the NT Authority\Local System account access to the share: https://serverfault.com/questions/135867/how-to-grant-network-access-to-localsystem-account/135874#135874

\\\\giishare\\prod\\gis\\software\\esri\\1091\\enterprise\\ArcGISGISServerAdvanced_ArcGISServer_1201091.prvc

Sometimes prvc files generated from My Esri can be incomplete or missing user information, which will cause the authorization to not complete, but no error will be thrown by the silent authorization wizard. Trying opening the prvc file with a text editor to ensure all the information is filled out. For example:

// User Information
First Name=ESRI
Last Name=Employee
Organization=ESRI
Department=Development
Email=employee@esri.com
Address 1=380 New York St.
City=Redlands
State/Province=CA
Location=United States
Location Code=US
Zip/Postal Code=92373
Phone Number=555-555-5555
Your Organization=Commercial/Private Business
Your Industry=Other
Yourself=Other
// Version=10.9

//Features and authorization numbers

ArcGIS Server=ECPXXXXXXXXX

Alternatively, you can try obtaining an .ecp file instead using the offline authorization option.

https://enterprise.arcgis.com/en/server/latest/install/windows/authorize-arcgis-server.htm#GUID-D8B5B450-A485-42EA-9BF5-AF4522C07AA3

  1. Launch the software authorization wizard and select "Authorize at Esri's website or by email to receive your authorization file."
  2. Follow through the additional prompts, this will then give you an authorization output file. Then you have two options:
    • Browse to the Esri Customer Service website (https://service.esri.com/drm). Follow the instructions on the screen to upload your authorization information file and download your .ecp authorization file.
    • Or, Send your authorization information file by email to authorize@esri.com. Esri will send your .ecp authorization file back to you in an email.

https://enterprise.arcgis.com/en/server/latest/install/linux/silently-install-arcgis-server.htm#ESRI_SECTION2_F12CF18F11F444AB90FD53CF86A054F2

Thanks, Cameron K.

ericandersondhshq commented 2 years ago

Thank you all very much for the suggestions, appreciate the support received so far.

I tested moving the license file locally on the target node - no change, same error I also ensured the PVRC file had all the user information filled out correctly and with no special characters

I tested the ECP license file instead, and am now getting a different error. I looked in the Event logs as indicated, but didn't see a clear point of failure. Any thoughts on how to debug this error further?

10/12/2022 10:35:45 AM: Exception calling "DownloadString" with "1" argument(s): "Unable to connect to the remote server" 10/12/2022 10:35:46 AM: The PowerShell DSC resource '[ArcGIS_ServerUpgrade]ServerConfigureUpgrade' with SourceInfo 'C:\Program Files\WindowsPowerShell\Modules\ArcGIS\Configurations-OnPrem\Upgrades\ServerUpgrade.ps1::337::13::ArcGIS_ServerUpgrade' threw one or more non-terminating errors while running the Test-TargetResource functionality. These errors are logged to the ETW channel called Microsoft-Windows-DSC/Operational. Refer to this channel for more details. 10/12/2022 10:35:46 AM: The SendConfigurationApply function did not succeed.

H47110815G commented 2 years ago

the needle in the haystack....

run powershell and test:

regards from Andreas

ericandersondhshq commented 2 years ago

Thanks again everyone for the support.

Using a ECP file was the key. The additional issues I noted in my comment above was related to ArcGIS Server continuously trying to install a Log4j patch every time the Windows service was started (even after machine reboot), which kept the ArcGIS Server site in a weird continuous restarting loop. I renamed the Windows folder where the patch install files were sourced and referenced by the Powershell .json file, and this resolved that.

Out of curiosity - what exactly is it about a ECP file that works better than a PRVC when working with Powershell? There were clearly no issues with the PRVC file itself since that authorized fine outside of the Powershell scripts.