Booz Allen's lean manufacturing approach for holistically designing, developing and fielding AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure operational deployment
Other
33
stars
8
forks
source link
Feature: Baton migration for ArgoCD template changes #313
Following #283, to deploy configuration store app before all other apps in Argocd, we have enabled both the Argocd sync wave and the auto sync for the children apps of the app so that after manually synchronizing the main app, the children app can be sync-ed automatically in the order defined by sync waves.
As part of that effort, we will include the migration script to enable the argocd sub-app auto sync feature on the templates so that when upgrade to baseline version 1.9.0, the apps can be deployed in the order defined in the sync wave.
DOD
Acceptance criteria required to realize the requested feature
[x] Write Baton migration script to update the downstream project argocd values.yaml to inlcude syncPolicy for create namespace, applyoutofsynconly, and automated for the sub app auto-sync
[x] Write Baton migration script to update the downstream project application templates to enable the syncPolicy configuration
BDD Scenario
Feature: Migrate a downstream project argocd template to include syncPolicy configuration
Scenario: Migrate a downstream project argocd template to include syncPolicy configuration
Verify the templates under -deploy/src/main/resources/templates/ has helm function to configure the syncPolicy
{{- with .Values.spec.syncPolicy }}
syncPolicy:
{{- toYaml . | nindent 4 }}
{{- end }}
Add to the fermenter-mda plugin executions in test-313-deploy/pom.xml
<execution>
<id>configuration-store</id>
<phase>generate-sources</phase>
<goals>
<goal>generate-sources</goal>
</goals>
<configuration>
<basePackage>com.boozallen.aissemble.test</basePackage>
<profile>configuration-store-deploy-v2</profile>
<!-- The property variables below are passed to the Generation Context and utilized
to customize the deployment artifacts. -->
<propertyVariables>
<appName>configuration-store</appName>
</propertyVariables>
</configuration>
</execution>
Run mvn clean install until all the manual actions are complete
Standup the test project within the argocd to ensure the configuration store is deployed before other resources and metastore-service-config configmap is updated with configuration store value
In the test-313-deploy/src/main/resources/app/hive-metastore-service/templates/configmap.yaml file:
add below content to the metadata:
labels:
aissemble-configuration-store: enabled
In the test-313/test-2213-deploy/src/main/resources/apps/hive-metastore-service/values.yaml file
replace fs.s3a.access.key value `123` with `$getConfigValue(groupName=aws-credentials;propertyName=AWS_ACCESS_KEY_ID)`
replace fs.s3a.secret.key value `456` with `$getConfigValue(groupName=aws-credentials;propertyName=AWS_SECRET_ACCESS_KEY)`
Download and unzip the attached 313-helper.zipand copy the src directory to the test-313 root directory
Modify the helm templates for Argocd deployment:
In the test-313-deploy/src/main/resources/values.yaml file, update the targetRevision and repo to reflect the correct value. e.g.:
targetRevision: main
repo: https://github.com/username/test-313
In the -deploy/src/main/resources/apps/configuration-store/values.yaml file, remove readOnly: true (ref: #309)
In the -deploy/src/main/resources/apps/configuration-store/ directory copy the content of the values-dev.yaml to the values-ci.yaml file
remove service and service.type from the values-ci.yaml file
service:
type: LoadBalancer
Mac User: In the -deploy/src/main/resources/apps/configuration-store/values-ci.yaml file update the volumePathOnNode to be /<pathToProject>/test-313/src/main/resources/configurations
Window User: In the -deploy/src/main/resources/values-ci.yaml file update the volumePathOnNode to be /mnt/c/Users/YOUR_USER/PATH/TO/283-helper/apps/configuration-store/src/main/resources/configurations
Create a repo for the project created
create a new test-313 repository from the github website
On the ArgoCD UI, verify that the test-313 app is shown
Sync test-313 app and verify that only the configuration-store app appears before the configuration-store status becomes healthy.
argocd app sync test-313 --revision main
Wait until the hive-metastore-service status is healthy and click into the hive-metastore-service -> metastore-service-config and verify the fs.s3a.access.key and fs.s3a.secret.key values are injected
## References/Additional Context
A clear and concise description of any alternative solutions or features you've considered.
Add any other context, links, or screenshots about the feature request here.
Description
Following #283, to deploy configuration store app before all other apps in Argocd, we have enabled both the Argocd sync wave and the auto sync for the children apps of the app so that after manually synchronizing the main app, the children app can be sync-ed automatically in the order defined by sync waves.
As part of that effort, we will include the migration script to enable the argocd sub-app auto sync feature on the templates so that when upgrade to baseline version 1.9.0, the apps can be deployed in the order defined in the sync wave.
DOD
Acceptance criteria required to realize the requested feature
BDD Scenario
Feature: Migrate a downstream project argocd template to include syncPolicy configuration Scenario: Migrate a downstream project argocd template to include syncPolicy configuration
Test Strategy/Script
How will this feature be verified?
Create a downstream project from baseline
v1.8.0
Add the SparkPipeline.json file to the
test-313-pipeline-models/src/main/resources/pipelines
directoryRun
mvn clean install
until all the manual actions are completeOnce the manual actions are complete, run
mvn clean install -Dmaven.build.cache.skipCache=true
once to get any remaining manual actionsUpgrade the project to
v1.9.0-SNAPSHOT
1.8.0
to1.9.0-SNAPSHOT
in theTiltfile
and rootpom.xml
mvn clean install -Dmaven.build.cache.skipCache
once to trigger the migration scriptVerify the
-deploy/src/main/resources/values.yaml
file has the sync policy configurationVerify the templates under
-deploy/src/main/resources/templates/
has helm function to configure thesyncPolicy
Add to the
fermenter-mda
plugin executions intest-313-deploy/pom.xml
Run
mvn clean install
until all the manual actions are completeStandup the test project within the argocd to ensure the configuration store is deployed before other resources and
metastore-service-config
configmap is updated with configuration store valuetest-313-deploy/src/main/resources/values.yaml
file, update thetargetRevision
andrepo
to reflect the correct value. e.g.:-deploy/src/main/resources/apps/configuration-store/values.yaml
file, removereadOnly: true
(ref: #309)-deploy/src/main/resources/apps/configuration-store/
directory copy the content of thevalues-dev.yaml
to thevalues-ci.yaml
file-deploy/src/main/resources/apps/configuration-store/values-ci.yaml file
update thevolumePathOnNode
to be/<pathToProject>/test-313/src/main/resources/configurations
-deploy/src/main/resources/values-ci.yaml file
update thevolumePathOnNode
to be/mnt/c/Users/YOUR_USER/PATH/TO/283-helper/apps/configuration-store/src/main/resources/configurations
git init
git add .
git commit -m "init test-283 base
git branch -M main
git remote add origin https://github.com/<username>/<repo_name>.git
git push -u origina main
helm install aissemble-infrastructure oci://ghcr.io/boozallen/aissemble-infrastructure-chart --version 1.9.0-SNAPSHOT --set jenkins.enabled=false --set ingress-nginx.enabled=false
argocd-server
to port8081
argocd admin initial-password -n argocd
(keep the password to login to argocd ui and cli console)http://localhost:8081/
and login as (admin/previous generated password)argocd login localhost:8081
(Use username: admin/password: previous generated password to login)test-313
app and verify that only theconfiguration-store
app appears before theconfiguration-store
status becomes healthy.hive-metastore-service
status is healthy and click into thehive-metastore-service
->metastore-service-config
and verify thefs.s3a.access.key
andfs.s3a.secret.key
values are injected