Feature: Create Baton migration to update project specific image name changes in downstream OSS projects

[colinpalmer-pro]

Description

In Issue-75, we updated the naming convention for project specific images generated in aiSSEMBLE downstream projects. This ticket will create a baton migration script to update the image names in downstream projects that were generated using any aiSSEMBLE version before 1.7.0. Impacted Images:

• airflow
• bom
• data-access
• inference
• jenkins
• kafka
• lineage-custom-consumer
• mlflow
• model-training-job
• policy-decision-point
• spark-operator
• spark-wroker-image
• vault
• versioning

DOD

• [x] Create baton migration (see bdd scenario below)
• [x] Update any documentation that references old image names for images listed above in the description
• [x] Update Release Notes

BDD Test Scenario

  Scenario: Upgrade aiSSEMBLE project specific image names
    Given a new downstream maven project "new-downstream-project"
    And a values file with the older project specific image naming convention
    When the project specific image name change migration executes
    Then the values file is updated to use new project specific image naming convention

  Scenario: Skipping image name migration if image does not contain the pattern "boozallen/<project-name>"
    Given a new downstream maven project "new-downstream-project"
    And a values file with a non project specific image
    When the project specific image name change migration executes
    Then the values file project specific image name change migration is skipped

Test Strategy/Script

ℹ️ In Progress

• Make sure the dev aiSSEMBLE branch is up-to-date and build the project (mvn clean install)
• Run the archetype plugin with the following commands to create a downstream project using the aiSSEMBLE version 1.6.1:

  
  mvn archetype:generate \
  -DarchetypeGroupId=com.boozallen.aissemble \
  -DarchetypeArtifactId=foundation-archetype \
  -DarchetypeVersion=1.6.1 \
  -DgroupId=com.boozallen.test \
  -DartifactId=issue-111 \
  -DprojectGitUrl=test-url \
  -DprojectName=issue-111

cd issue-111

- Create a json file, `SimpleDataDeliveryExample.json` in issue-111-data-delivery/issue-111-data-delivery-pipeline-models/src/main/resources/pipelines and add the following contents:

{ "name":"SimpleDataDeliveryExample", "package":"com.boozallen.aissemble.documentation", "type":{ "name":"data-flow", "implementation":"data-delivery-spark", "executionHelpers": ["airflow"] }, "steps":[ { "name":"IngestData", "type":"synchronous", "dataProfiling":{ "enabled":false } } ] }

- Create a json file, `ExampleMachineLearningPipeline.json` in issue-111-data-delivery/issue-111-data-delivery-pipeline-models/src/main/resources/pipelines and add the following contents:

{ "name": "ExampleMachineLearningPipeline", "package": "com.boozallen.aissemble", "type": { "name": "machine-learning", "implementation": "machine-learning-mlflow" }, "steps": [ { "name": "AissembleMachineLearningTraining", "type": "training", "inbound": { "type": "messaging", "channelName": "train" } }, { "name": "AissembleMachineLearningInference", "type": "inference" } ] }

- Ensure you are connected to the vpn and build the project from the root - `mvn clean install`
- Perform the manual actions and rebuild (`mvn clean install`) until there are no more manual actions
- Update the root pom.xml build-parent version to `1.7.0-SNAPSHOT` 
- Remove the following line from the issue-111-deploy pom.xml
  - `<version>${version.clean.plugin}</version>`
- Run `mvn clean install -DoldHelmRepositoryUrl=<old-helm-url>`
  - Can get the old helm chart url from the s3-local chart
  - The build will fail at the simple data delivery pipeline due to a new migration (We are figuring out the best guidance to handle the migration to the pom file). Kick the build off again: `mvn clean install -rf :simple-data-delivery-example -DoldHelmRepositoryUrl=<old-helm-url>`
- Inspect the following files (all lines inspected below should not have "boozallen" except the hive-metastore-service):
  - `issue-111-deploy/src/main/resources/apps/airflow/values.yaml`
    - Validate that the `defaultAirflowRepository` field has`.../aiops/issue-111-airflow-docker` **without** `boozallen/`
  - `issue-111-deploy/src/main/resources/apps/aissemble-machine-learning-inference/values.yaml`
    - Validate it contains the line: `name: issue-111-aissemble-machine-learning-inference-docker`
  - `issue-111-deploy/src/main/resources/apps/aissemble-machine-learning-training-image/aissemble-machine-learning-training-image.yaml`
    - Validate it contains the line: `image: issue-111-aissemble-machine-learning-training-docker`
  - `issue-111-deploy/src/main/resources/apps/hive-metastore-service/values.yaml`
    - Validate it contains the line: `name: boozallen/aissemble-hive-service`
  - `issue-111-deploy/src/main/resources/apps/policy-decision-point/values.yaml`
    - Validate it contains the line: `name: issue-111-policy-decision-point-docker`
  - `issue-111-deploy/src/main/resources/apps/spark-worker-image/spark-worker-image.yaml`
    - Validate it contains the line: `image: issue-111-spark-worker-docker`
  - `issue-111-pipelines/simple-data-delivery-example/src/main/resources/apps/simple-data-delivery-example-base-values.yaml`
    - Validate it contains the line: `image: "issue-111-spark-worker-docker:latest"` 
  - `TiltFile`
    - Search for `boozallen/` and validate there are no results

[jacksondelametter]

All tests pass, ticket is complete.