Open df-sloughton opened 7 months ago
@df-sloughton, can you share the contents of /opt/arcgis/portal/.ESRI_P_PATCH_LOG
? I am curious if the Portal for ArcGIS 10.9.1 Security 2024 Update 1 Patch is listed there or not.
I ran through this and was not able to replicate the issue.
Chef Debug Run Log:
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-Log4j-PatchB-linux.tar] action install[2024-04-11T09:30:51-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-Log4j-PatchB-linux.tar] action install (arcgis-enterprise::install_patches line 41)
[2024-04-11T09:30:51-07:00] INFO: Installing '/opt/software/archives/patches/ArcGIS-1091-PFA-Log4j-PatchB-linux.tar' patch...
[2024-04-11T09:30:51-07:00] DEBUG: "/tmp/d20240411-364944-ghutm6/PFA-1091-P-721B/applypatch" -s -portal
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar] action install[2024-04-11T09:31:04-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar] action install (arcgis-enterprise::install_patches line 41)
[2024-04-11T09:31:04-07:00] INFO: Installing '/opt/software/archives/patches/ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar' patch...
[2024-04-11T09:31:04-07:00] DEBUG: "/tmp/d20240411-364944-evpphz/PFA-1091-P-931B/applypatch" -s -portal
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-QCS-Patch-linux.tar] action install[2024-04-11T09:34:28-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-QCS-Patch-linux.tar] action install (arcgis-enterprise::install_patches line 41)
[2024-04-11T09:34:28-07:00] INFO: Installing '/opt/software/archives/patches/ArcGIS-1091-PFA-QCS-Patch-linux.tar' patch...
[2024-04-11T09:34:28-07:00] DEBUG: "/tmp/d20240411-364944-rxyy5h/PFA-1091-P-805/applypatch" -s -portal
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar] action install[2024-04-11T09:39:08-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar] action install (arcgis-enterprise::install_patches line 41)
[2024-04-11T09:39:08-07:00] INFO: Installing '/opt/software/archives/patches/ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar' patch...
[2024-04-11T09:39:10-07:00] DEBUG: "/tmp/d20240411-364944-f97yz7/PFA-1091-P-885B/applypatch" -s -portal
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install[2024-04-11T09:45:07-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install (arcgis-enterprise::install_patches line 41)
[2024-04-11T09:45:07-07:00] INFO: Installing '/opt/software/archives/patches/ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar' patch...
[2024-04-11T09:45:09-07:00] DEBUG: "/tmp/d20240411-364944-bgo8b2/PFA-1091-P-991/applypatch" -s -portal
Contents of .ESRI_P_PATCH_LOG:
$ cat /opt/arcgis/portal/.ESRI_P_PATCH_LOG
#START
QFE_ID: PFA-1091-P-991
QFE_TYPE: Patch
QFE_TITLE: Portal for ArcGIS 10.9.1 Security 2024 Update 1 Patch
QFE_FILE: ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar
INSTALL_TIME: 04/11/24 09:46:37
#END
#START
QFE_ID: PFA-1091-P-885B
QFE_TYPE: Patch
QFE_TITLE: Portal for ArcGIS 10.9.1 Security 2023 Update 1 Patch B
QFE_FILE: ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar
INSTALL_TIME: 04/11/24 09:40:11
#END
#START
QFE_ID: PFA-1091-P-805
QFE_TYPE: Patch
QFE_TITLE: Portal for ArcGIS 10.9.1 QuickCapture Security Patch
QFE_FILE: ArcGIS-1091-PFA-QCS-Patch-linux.tar
INSTALL_TIME: 04/11/24 09:34:33
#END
#START
QFE_ID: PFA-1091-P-931B
QFE_TYPE: Patch
QFE_TITLE: Portal for ArcGIS 10.9.1 Enterprise Sites Security Patch B
QFE_FILE: ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar
INSTALL_TIME: 04/11/24 09:31:24
#END
#START
QFE_ID: PFA-1091-P-721B
QFE_TYPE: Patch
QFE_TITLE: Portal for ArcGIS 10.9.1 Log4j Patch B
QFE_FILE: ArcGIS-1091-PFA-Log4j-PatchB-linux.tar
INSTALL_TIME: 04/11/24 09:31:02
#END
Patch Notification Result
$ /opt/arcgis/portal/tools/patchnotification/patchnotification
Unable to access display. Switching to console mode.
================================================================================
ArcGIS Enterprise Patch Notification
================================================================================
Installed Components
Portal for ArcGIS 10.9.1
================================================================================
Available Updates
Portal for ArcGIS
(no updates available)
================================================================================
Installed Patches
- Portal for ArcGIS 10.9.1 Security 2024 Update 1 Patch
Installed: 4/11/24, 9:46 AM
- Portal for ArcGIS 10.9.1 Security 2023 Update 1 Patch B
Installed: 4/11/24, 9:40 AM
- Portal for ArcGIS 10.9.1 QuickCapture Security Patch
Installed: 4/11/24, 9:34 AM
- Portal for ArcGIS 10.9.1 Enterprise Sites Security Patch B
Installed: 4/11/24, 9:31 AM
- Portal for ArcGIS 10.9.1 Log4j Patch B
Installed: 4/11/24, 9:31 AM
================================================================================
To browse a full list of Esri patches and service packs, visit the Esri Support site:
http://support.esri.com/Downloads
Thanks Nope not there. That install time is the image building process and the order looks correct. Also where are the other patches?
QFE_ID: PFA-1091-P-931B QFE_TYPE: Patch QFE_TITLE: Portal for ArcGIS 10.9.1 Enterprise Sites Security Patch B QFE_FILE: ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar INSTALL_TIME: 04/08/24 07:03:20
QFE_ID: PFA-1091-P-805 QFE_TYPE: Patch QFE_TITLE: Portal for ArcGIS 10.9.1 QuickCapture Security Patch QFE_FILE: ArcGIS-1091-PFA-QCS-Patch-linux.tar INSTALL_TIME: 04/08/24 07:02:20
QFE_ID: PFA-1091-P-885B QFE_TYPE: Patch QFE_TITLE: Portal for ArcGIS 10.9.1 Security 2023 Update 1 Patch B QFE_FILE: ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar INSTALL_TIME: 04/08/24 07:01:25
QFE_ID: PFA-1091-P-721B QFE_TYPE: Patch QFE_TITLE: Portal for ArcGIS 10.9.1 Log4j Patch B QFE_FILE: ArcGIS-1091-PFA-Log4j-PatchB-linux.tar INSTALL_TIME: 04/08/24 06:52:27
Hi @df-sloughton,
Are you using Chef to download the patches or are the patches being downloaded/staged by another means? Looks like there are 3 patches that have been made obsolete and replaced with new ones:
ArcGIS-1091-PFA-SEC2022U1-Patch-linux.tar
ArcGIS-1091-PFA-SEC2022U2-PatchB-linux.tar
ArcGIS-1091-PFA-ESFD-Patch-linux.tar
So that leaves us with these 5 patches:
Chef will check the /opt/arcgis/portal/.ESRI_P_PATCH_LOG
file to see if the patch has previously been installed, if found then it skips the patch installation. I am definitely not seeing Portal for ArcGIS 10.9.1 Security 2024 Update 1 Patch
within your .ESRI_P_PATCH_LOG so I am not sure why its being skipped or not being installed.
However, when you tried manually applying the patch it does throw this error which is interesting:
No $HOME/.ESRI.properties files found.
I believe it is looking for /home/arcgis/.ESRI.properties.ip-xxxxxyyyyy.10.9.1
. Does this file exist and if so what is its contents?
I notice the Target OS is Ubuntu 22.02. Both the cookbooks v4.0.0 and ArcGIS Enterprise 10.9.1 are not supported/certified on Ubuntu 22.04 LTS:
Perhaps the issue stems from something weird or different with Ubuntu 22. Can you try on Ubuntu 20.04 LTS?
Thanks, Cameron K.
Hi @df-sloughton,
I think we may have figured out why the ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar patch is being skipped or not installed. When I remove ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar file from my patch directory Chef logs the following message:
* arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install[2024-04-12T13:47:21-07:00] INFO: Processing arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install (arcgis-enterprise::install_patches line 41)
(up to date)
This is the same message you are receiving. It should actually log a warning or message that the file was not found.
In your json config file can you ensure the following is added: https://github.com/Esri/arcgis-cookbook/blob/a6140542a084fc81e90bfa2e8e9d97a734651c7c/templates/arcgis-portal/10.9.1/linux/arcgis-portal-patches-apply.json#L5-L7
Otherwise it is going to look for the patch in the default location of /opt/software/esri/patches
which in your case may not exist. Feel free to change the path in the json and ensure the patch file exists in the location specified. It should then install successfully.
I had both "patches" and "local_patches" in an attempt to download patches on the fly rather than have them pre-downloaded. Its not clear what the relationship is. Local_patches seems to be put into internal param patches_dir which is a param into download_patches function, and has worked for our image building thus far: https://github.com/search?q=repo%3AEsri%2Farcgis-cookbook%20patches_dir&type=code The folder does not exist initially, but its parent does, and chef creates it just before downloading: https://github.com/search?q=repo%3AEsri%2Farcgis-cookbook+%27local_patches%27&type=code
My patch order was by date, I am yet to try your order above. I had another issue elsewhere on Linux where the order was important.
"arcgis": {
"version": "10.9.1",
"run_as_user": "arcgis",
"repository": {
"archives": "/opt/software/archives",
"setups": "/opt/software/setups",
"local_patches": "/opt/software/archives/patches",
"patches": "/opt/software/archives/patches",
"patch_notification": {
"products": [
"ArcGIS Enterprise, Portal for ArcGIS"
]
}
},
"portal": {
"wa_name": "portal",
"install_dir": "/opt",
"configure_autostart": true,
"install_system_requirements": true,
"patches": [
"ArcGIS-1091-PFA-Log4j-PatchB-linux.tar",
"ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar",
"ArcGIS-1091-PFA-QCS-Patch-linux.tar",
"ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar",
"ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar"
]
}
}
2024-04-14T23:34:13Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-Log4j-PatchB-linux.tar] action install
2024-04-14T23:34:13Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu:
2024-04-14T23:44:05Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar] action install
2024-04-14T23:44:05Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu:
2024-04-14T23:44:55Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-QCS-Patch-linux.tar] action install
2024-04-14T23:44:55Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu:
2024-04-14T23:45:45Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar] action install
2024-04-14T23:45:45Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu:
2024-04-14T23:45:45Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install (up to date)
"arcgis": {
"version": "10.9.1",
"run_as_user": "arcgis",
"repository": {
"archives": "/opt/software/archives",
"setups": "/opt/software/setups",
"patches": "/opt/software/archives/patches",
"patch_notification": {
"products": [
"ArcGIS Enterprise, Portal for ArcGIS"
]
}
},
"portal": {
"wa_name": "portal",
"install_dir": "/opt",
"configure_autostart": true,
"install_system_requirements": true,
"patches": [
"ArcGIS-1091-PFA-Log4j-PatchB-linux.tar",
"ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar",
"ArcGIS-1091-PFA-QCS-Patch-linux.tar",
"ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar",
"ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar"
]
}
}
2024-04-13T10:53:31Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-Log4j-PatchB-linux.tar] action install (up to date)
2024-04-13T10:53:31Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar] action install (up to date)
2024-04-13T10:53:31Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-QCS-Patch-linux.tar] action install (up to date)
2024-04-13T10:53:31Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar] action install (up to date)
2024-04-13T10:53:31Z: 1_arcgis_ubuntu.amazon-ebs.arcgis_portal_ubuntu: * arcgis_enterprise_patches[Install patch ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar] action install (up to date)
I tried on Ubuntu 20.04 - same result.
I interrupted the build and had a look at the /opt/software/archives/patches folder and it simply does not download it. I did not run out of room either, df reported 16GB free still.
# /opt/software/archives/patches# ls
ArcGIS-1071-PFA-SEC2023U1-PatchB-linux.tar ArcGIS-1081-PFA-SEC2023U1-PatchB-linux.tar ArcGIS-1091-PFA-Log4j-PatchB-linux.tar ArcGIS-110-PFA-SEC2023U1-Patch-linux.tar
ArcGIS-1081-PFA-ESSEC-PatchB-linux.tar ArcGIS-109-PFA-QCS-Patch-linux.tar ArcGIS-1091-PFA-QCS-Patch-linux.tar ArcGIS-111-PFA-ESSEC-PatchC-linux.tar
ArcGIS-1081-PFA-QCS-Patch-linux.tar ArcGIS-1091-PFA-ESSEC-PatchB-linux.tar ArcGIS-1091-PFA-SEC2023U1-PatchB-linux.tar
Chef uses https://downloads.esri.com/patch_notification/patches.json I checked that and cannot see why it would not pick it up / how chef arcgis cookbooks 4.0.0 filter for it other than filename prefix. I downloaded it manually from the URL it would use and it worked, and the verify SHA256sum worked also.
{
"Name": "Portal for ArcGIS Security 2024 Update 1 Patch",
"Products": "Portal for ArcGIS",
"Platform": "Linux,Windows",
"url": "https://support.esri.com/en-us/patches-updates/2023/portal-for-arcgis-security-2024-update-1-patch",
"QFE_ID": "PFA-1081-P-990",
"ReleaseDate": "04/04/2024",
"Critical": "security",
"PatchFiles": [..."https://gisupdates.esri.com/QFE/PFA-1091-P-991/ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar"...],
"SHA256sums": [..."ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar:9B1464F497F88770AE033A2AABF2F055870321FF9D98D1E095CF609F9E21029F"...],
"MD5sums": []
}
wget https://gisupdates.esri.com/QFE/PFA-1091-P-991/ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar
--2024-04-15 03:54:38-- https://gisupdates.esri.com/QFE/PFA-1091-P-991/ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar
Resolving gisupdates.esri.com (gisupdates.esri.com)... 104.74.27.7
Connecting to gisupdates.esri.com (gisupdates.esri.com)|104.74.27.7|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 698920960 (667M) [application/x-tar]
Saving to: ‘ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar’
ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar 100%[===================================================================================================================================>] 666.54M 60.9MB/s in 11s
2024-04-15 03:54:51 (60.2 MB/s) - ‘ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar’ saved [698920960/698920960]
# echo "9B1464F497F88770AE033A2AABF2F055870321FF9D98D1E095CF609F9E21029F ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar" > sha256sums.txt
# sha256sum -c sha256sums.txt
ArcGIS-1091-PFA-SEC2024U1-Patch-linux.tar: OK
@df-sloughton
Could you try using cookbooks v4.2.0?
I believe there was a regression introduced in v4.0.0 with the patch download filtering when a change was made to the "Products" field within https://downloads.esri.com/patch_notification/patchesPretty.json.
I think this could be why the patch is not downloading.
Thanks, Cameron K.
I have not tested 4.2.0 yet. But the workaround if yo uhave to patch manually is create the $HOME/esri.properties.* file:
$ vi $HOME/.ESRI.properties.ip-<192-168-1-1>.10.9.1
Replace <192-168-1-1> with your IP address with hyphens, not dots, and no angular brackets.
Paste and save into the file something like this:
#Mon Apr 15 03:23:17 UTC 2024
Z_REAL_VERSION=10.9.1
Z_ArcGISPortal_INSTALL_DIR=/opt/arcgis/portal
ARCLICENSEHOME=
ESRI_PROGRAM_FILES=
Z_ArcGISPortal_WebStyles_INSTALL_DIR=/opt/arcgis/portal
Z_WebStyles_VERSION=10.9.1
then run the patch process or download and install each missing patch manually:
$ /opt/arcgis/portal/tools/patchnotification/patchnotification -i all
Hi @df-sloughton
The $Home/.ESRI.properties file gets created by the portal setup during installation. It must've been there at some point otherwise I would expect the other patches to not have installed as well.
Is this an environment in AWS? We have some logic in the cookbooks system recipe to rename the .ESRI.properties file in aws specifically when it comes to images. For example, if portal is installed and an AMI is created then that .ESRI.properties file will contain the hostname/ip of the original machine where the image was created. Then when that AMI is used in a new EC2 instance that has a different ip/hostname the file needs to be renamed to match. Perhaps something got clobbered during this process causing the file to disappear:
I would be interested in the full chef debug logs if you have them.
Thanks, Cameron K.
I have not been able to test on 4.2.0+ yet, however an update - the Chrome/Edge 127 patch works on 4.0.0, namely ArcGIS-1091-PFA-AD-Patch-linux.tar. However if you then manually run patchnotification, the now older problematic patch will install out of order to this new patch, and you will have a mixed environment. We had users not seeing Chrome/Edge 127 issues and some did see issues, until I uninstalled both manually and installed them in order manually.
Hi Im interested in all the patches for Portal on Linux, including getting past any recent patch installer/Enterprise Sites patching issues if they exist on Linux. Packer + Chef seem to say it installed, or was already there (says "upto date")? But when deployed image patchnotification says its available still to install.
Target OS: Ubuntu Linux 22.02 ArcGIS Enterprise 10.9.1 Chef ArcGIS Cookbooks 4.0.0 packer:1.9.1 on GitHub Cloud > Actions Runner
Chef JSON snippet (trying also to get the order right):
Chef logs via Packer via Github Action Runner snippet
Patchnotification output:
This might be another problem or related, but right now I cant manually patch either: