Complete preparation for install of ArcGIS Enterprise Portal in GCP

[nathankota]

System Diagram

Statement of Need/Objective

ArcGIS Enterprise Portal is required to coordinate with BLM and SITLA to update and maintain the BLM PLSS fabric, within the Esri parcel fabric model. Additionally, Enterprise will be used to improve collaboration with counties and other local governments around data aggregation workflows including, but not limited to, roads centerlines, voting-related data, and so forth. The project goal is implementation of ArcGIS Enterprise/Portal within the UGRC GCP environment for use with the PLSS (parcel) fabric and branch versioning.

Benefit Hypothesis

Installation of Enterprise Portal in the cloud to enable the plss fabric and other use cases.

Acceptance Criteria

• Necessary licensing obtained
• Review and complete Jumpstart docs
• Meet internally to decide on hardware/architecture (distributed is preferred/advised)
• Schedule and participate in Esri Jump Start for ArcGIS Enterprise Portal installation in GCP
• Acceptance testing/validation of the complete Enterprise Portal implementation

[nathankota]

Future Features

• Create branch versions
• Load data
  • Ref: #24 (PLSS)
  • Roads
• Connect parcel fabric to SITLA and BLM's process
• Test the successful editing of road centerlines with one of the stewards (WFRC, Moab City, Grand County)

[nathankota]

Sprint 1 Tasks

• [x] Meet and complete system diagram and submit to Esri (@MichaelAGRC, @steveoh)
• [x] Ding Jan about ports and send him the current diagram (@MichaelAGRC)
• [x] Request version 11.1 license to replace the existing 10.9.1 license we asked for initially (@nathankota)
  • Emailed Amber on July 17, 2023
  • Looks like we only have 11.1 licenses now

[nathankota]

Sprint 2 Tasks:

• [x] Michael get some time with Steve, Scott, (maybe Nathan) to determine next sequence of events
• [x] GCP infrastructure setup (machines, db, firewalls all there for dev)
• [x] Firewall requests
• [ ] ~Service accounts~
• [ ] ~Setup enterprise geodatabase (cloud sql - for roads and fabric data)~
• [ ] ~SSL cert - figure out what to do about this~

[steveoh]

Sprint 2 Updates

• I terraformed the dev portal infrastructure and asked @MichaelAGRC to test the ports and review the specs
• I recommend installing an enterprise geodatabase into the cloud sql we stood up to see how it goes (@MichaelAGRC)
• I created an incident with cloud core to join it to the utah ad that is getting assigned to the wrong groups and should sort it self out soon.

[nathankota]

Sprint 3 Tasks

• [ ] validate ports and review specs of dev infrastructure
• [ ] ~license and install enterprise gdb in existing cloud sql and validate it works as expected~
• [x] get dev machines sys prepped and joined to the shared VPC and Utah domain to test utah\service-account for installation
• [ ] ~obtain cert (if prod infrastructure is completed by end of sprint)~

[steveoh]

Sprint 3 updates

Aug 9

• Switching the dev project to use the shared vpc to join the utah domain is going to cause some pain
  1. We will no longer own the project with the firewall rules and will need to go through the standard dts firewall process and wait the normal amount of time for each rule to be approved and implemented.
  2. Since the firewalls are the biggest unknown and will take the most time to update, we may have issues with the kick start timeline. @MichaelAGRC is the kick start 3 consecutive days or can it be split up to allow for dts processes to run their course? _--Esri answered with a polite no. In their words "When we start we want to do all 3 days consecutively."_
  3. The current firewall testing cannot continue since I have started to remove the rules to migrate to the shared vpc project. Installing SDE can continue but will most likely require some permission tweaking and a bit of training to gain access to the database.

I believe we can purchase a certificate or have one provisioned prior to the prod infrastructure being completed. We can also reserve the DNS for portal and server.

Aug 10

• enterprise firewalls requested (RITM0249285) you can review the attached rules if you like

Aug 16 6:32PM

• machines have been renamed and joined to the domain
• still waiting on everything to shake out from installing all the AV, forescout, and zabbix
• amy is testing whether AD firewall rules are open on the prod shared vpc

Aug 21 12:17PM

• portal and AD firewall rules are passed approvals
• required software firewalls are in approval. The required software is Nxlog, Comodo, Rubrik backups, Zabbix, and Forescout
• prod subnet has AD firewalls already open and firewall to open for software is put in.
• still waiting to hear from michael about questions from the 9th
• requested service account INC1876710

[nathankota]

Sprint 4 Tasks

• [x] Consider ArcGIS Monitor. Do we need it?
• [ ] validate ports and review specs of dev infrastructure
• [ ] license and install enterprise gdb in existing cloud sql and validate it works as expected
  • database credentials in bitwarden (ut-dts-agrc-portal-dev sql)
• [ ] can we order a cert? or do we need name and IP first?
• [ ] get in touch with Esri Jumpstart folks with questions, concerns, considerations
• [ ] Schedule the Jumpstart

[nathankota]

Sprint 4 Notes

August 23

Let's skip Monitor during the initial portal install, and I'll schedule time with Esri to learn more about it in Q2

Bringing this down from above... Since the firewalls are the biggest unknown and will take the most time to update, we may have issues with the kick start timeline. @MichaelAGRC is the kick start 3 consecutive days or can it be split up to allow for dts processes to run their course? --Esri answered with a polite no. In their words "When we start we want to do all 3 days consecutively."

August 29

Michael tested ports and rejected the firewall requests RITM0249285

August 29th

Cameron Taylor, Aug 29, 4:35 PM Thank you, ServiceNow has generated a ticket and we will start looking into that asap.

[nathankota]

Sprint 5

• [x] Obtain cert INC1884544
• [x] Work through firewall rejection and validate ports INC1881434
• [x] Enterprise GDB (SDE) installation
  • Install Pro and Sql Server Mgt Studio on data store machine
  • database credentials in bitwarden (ut-dts-agrc-portal-dev sql)

[MichaelAGRC]

Sprint 5 Notes:

9/7/2023 - Michael says: Database testing was successful. Was able to connect from the datastore server using both SSMS and ArcPro. Was able to create users and assign privileges. Was also able to create an enterprise geodatabase and add data to it.

[steveoh]

Steve Sprint 5 Notes

Sept 7 4:42pm Summary

• The terraform is 99% complete with regard to creating the 3 machines, load balancer, and database. There are a couple of labels I need to add for dts firewalls.
• All DTS firewall rules should exist in the prod and dev shared vpc to join the prod compute vm's to the domain without delay other than their work to install and configure the software.
• The portal machine firewall rules weren't great and I will review and update them with someone from the core cloud team.

Sept 8 11:46am Update

• machines are terraformed
• database is available

Sept 11 5:40PM

• the machines are mostly joined to the domain with software installed. Amy was confused about something but I think it will get sorted out
• Firewall rule is in RITM0252307 for enterprise. I used GCP networking tags so we'll see if it passes security.

Sprint Tasks

• [x] create prod project
• [x] format external drives
• [x] request vpc join
• [x] request domain join
• [x] submit firewall rules for portal
• [x] install db software on datastore
• [x] put cert on the portal machine? load balancer?
• [x] put portal installer and licenses on machine
• [x] put server installer and license on machine
• [x] put data store installer and license on machine
• [x] create prod egdb
• [x] schedule kickoff
• [x] schedule jumpstart

[nathankota]

Nathan Sprint 5 Notes

September 8, 11:00 a.m.

• A cert has been provided via Virtru secure email for arcgisportal.gis.utah.gov, according to INC1884544.
• I see in the tasks above a question about a load balancer cert. I asked Rachel G., who provided the cert, whether we need another for the load balancer. Her response was:

  It depends. Does this load balancer require that you use their CSR? If so, then another certificate needs to be issued. If not, then the same certificate can be used as long as it is for the same "common name".

@MichaelAGRC, @steveoh any input or guidance you can provide about cert(s) next steps is greatly appreciated.

[steveoh]

Sprint 6 Notes (Steve)

Sept 20 3:20pm

Had the kick off. Waiting for the ports from esri to double check firewalls. Our action items are mainly in the following incidents.

• SAML discovery INC1892877
• Antivirus disable request INC1892915

[nathankota]

The Jumpstart meeting with Esri is scheduled for October 24-26. We are following the last couple of action items prior to the installation via #213