search

bcgov / digital-journeys

PSA Forms System
https://bcgov.github.io/digital-journeys/
Apache License 2.0
8 stars 7 forks source link

Tool: Troubleshoot and Fix Patroni Defective Image #1562

Closed iman-jamali-fw closed 9 months ago

iman-jamali-fw commented 9 months ago

The latest version of the Patroni image, patroni-postgres:12.4-latest, is defective. This issue is causing service interruptions in associated components namely Keycloak, API, and Camunda.

A functioning version of the Patroni image is stored in the BC Gov Artifactory, accessible at artifacts.developer.gov.bc.ca/bcgov-docker-local/patroni-postgres:2.0.1-12.4-latest. Warren has successfully reconfigured the Patroni StatefulSet to direct towards this BC Gov working image. However, there is an ongoing risk.

The Patroni ImageStream in OpenShift is still linked to the defective Patroni image, patroni-postgres:12.4-latest. This poses a potential problem during a rebuild/restart of the Patroni StatefulSet (might cause by OpenShfit Cluster updates), as new pods could be created based on the defective image.

Acceptance Criteria:

iman-jamali-fw commented 9 months ago

Restarting Patronin Statefulset on DEV worked well and the new pods were built off the BC Gov hosted image (working image). Will continue troubleshooting on TEST for today.

iman-jamali-fw commented 9 months ago

Tested out Patroni in TEST and pods were recrated (on oc rollout restart) by the BC Gov hosted image and working fine.

Stella-Archer commented 9 months ago

To test: check Mat/Pat check all service are working 'end to end ' check data in CRM

Stella-Archer commented 9 months ago

@bhumin-fw - End to end mat/pat looks great TEST CRM received string for details and PDF attachments. However I did notice the Subject field in the CRM generated ticket has reverted back to the value we had before our last deploy to PROD - could have nothing to do with the Patroni issue but want to check and get it fixed anyways OLD ticket with correct info in subject field

image.png

NEW ticket with TEST back in the subject Field

image.png
bhumin-fw commented 9 months ago

@Stella-Archer on test environment it is still "TEST Maternity, Parental Leave and Allowance form for ". That value did not change or update. We did make change in PROD only for CRM subject and remove "TEST" prefix from the subject. On PROD environment it is "Maternity, Parental Leave and Allowance form form ".

No this is not related to patroni database issue.