Find path to better deployment automation

[devopsmatt]

Create a list of manual and brittle steps, pain points. Identify steps that can be reasonably automated. Create a resolution proposal.

DoD: Resolution proposal addressing all automatable gaps in the current deployment automation in a Google doc

[emvaldes]

I have reached out to Jack Wang and asked for a time to meet so that we can work on setting up my environment as we talked about in our previous session. At this point in time, he has provided me with some information so I can follow the setup process on my own.

This is the information he has provided me with:

[emvaldes]

Good morning. If you haven't already, I would start by following the documentation here. If there's something that needs clarity, let me know. README.md Getting Started as a ReportStream Developer Overview These pages are a work in progress, please see the old pages if something is
missing Table of contents • Table of contents • Locally installed software prerequisites • Bulding the baseline
• First build
• Seed the Postgres db and vault • Running ReportStream backend • Debugging • Running the frontend • Running the static site • Submit a sample message through the app • Troubleshooting • Next steps Locally installed software prerequisites You will need to have at least the following pieces of software installed locally in order to be able to build and/or
debug this project: • git including git-bash if you're on Windows • Docker or Docker Desktop • OpenJDK (currently targetting 17)
• See the linked docs for important instructions on setting JAVA_HOME • Azure Storage Explorer • An IDE. IntelliJ is recommended for Kotlin/debugging support. To reduce the build-debug cycle time you can install these tools to run the code directly. These tools are required if
you are using an Apple Silicon computer, otherwise they are optional. • Azure Functions Core Tools (currently targetting 4) • AzureCLI • Gradle • One or more PostgreSQL Clients Warning If you are using an Apple Silicon computer you will need to use Gradle as Microsoft's Azure Function Docker image is
only amd64 compatible. There are
also known issues
running Intel Docker images on Apple Silicon. Both approaches to running the project are documented here.
Many of our local developer tools are set up to run in Docker containers. Looking at our docker-compose.yml file,
the prime_dev service does not work on Apple Silicon. Likewise, the builder service in
our docker-compose.build.yml does not work. Bulding the baseline First build The cleanslate.sh script does the base work needed to start developing for ReportStream. It only needs to be run once.
This script runs on Apple processors, but it skips a few steps. We will need to do these missing steps by hand.

1. Clone the prime-reportstream repository
   to your workstation using git.
2. Initialize your environment and run an initial build by running the following command using a Linux shell.
   Note you can run cleanslate.sh script to reset your environment as well (run ./cleanslate.sh --help for more
   information). The cleanslate.sh script not only cleans, but also performs a first build and setup of all artifacts
   needed for the build cd ./prime-router ./cleanslate.sh Docker Desktop* If you are using Docker Desktop, verify that it is running prior to building or running ReportStream locally. Gradle/Apple Silicon Run cleanslate.sh

   build the project

   ./cleanslate.sh --verbose

   ...

   Check that a Postgres instance is running

   docker ps

   CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

   2962fb214203 postgres:11 "docker-entrypoint.s…" 57 minutes ago Up 57 minutes 0.0.0.0:5432->5432/tcp prime-router_postgresql_1

   Run support services ReportStream depends on set of services to be up before running the main Azure service. The cleanslate.sh script
   starts a Postgres database but skips starting a few more that are otherwise started by default when cleanslate.sh
   start is run on a non-Apple processor: • Azurite - a simulator of Azure storage • Vault - a secret store • SFTP - an SFTP server • soap-webservice - SOAP web service emulator docker compose -f docker-compose.build.yml up --detach Additionally, to ensure that Vault is running and the credentials are stored correctly, run the following (which is
   normally covered by cleanslate.sh on non-Apple processors): docker compose up --detach vault 1>/dev/null 2>/dev/null You can take down these services by running ./gradlew composeDown or docker compose down command.
   For now, leave these services running and open up a new terminal session. Seed the Postgres db and vault To run tests, the Postgres DB and the credential vault need to be seeded with values.
   We will need to have ReportStream running for these steps to work (see previous steps).
   Again, we will use a Gradle task to do these steps. Warning If you are getting a connection error during the reloadTables step, you're probably on a mac with the custom Apple
   CPU. Use command gradlew run to get the app up and running and try again. ./gradlew primeCLI --args "create-credential --type=UserPass --persist=DEFAULT-SFTP --user foo --pass pass" ./gradlew reloadTables ./gradlew reloadSettings Running ReportStream backend Verify that your docker engine is running prior to building or running ReportStream locally (either Docker Desktop or
   through the command line). Building and running ReportStream requires a locally accessible PostgreSQL database instance that is initially setup and
   run by the cleanslate.sh script. This database instance runs as a Docker container defined by
   the docker-compose.build.yml file. You will need to start this database instance upon a workstation reboot by using
   the following command: cd ./prime-router docker compose --file "docker-compose.build.yml" up --detach Gradle/Apple SiliconUse Gradle to launch ReportStream, as it will set up the environment variables that ReportStream needs. If you are on a Mac with the custom Apple CPU and are following this guide step by step you've already done this step when you seeded Postgres db and vault. ./gradlew run ReportStream should continue to run after launching. A successful build will appear like so: ===========-- 90% EXECUTING [46s]

   :azureFunctionsRun

A ctrl-c will escape the running ReportStream instance. Tip The recommended way to rapidly develop on the ReportStream backend is to use `./gradlew q…