boozallen / aissemble

Booz Allen's lean manufacturing approach for holistically designing, developing and fielding AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure operational deployment
Other
34 stars 8 forks source link

Feature: aiSSEMBLE-specific Hive MySQL image should be deprecated #330

Closed ewilkins-csi closed 1 month ago

ewilkins-csi commented 1 month ago

Description

With the introduction of the v2 Spark Infrastructure chart, we've officially moved the default Hive Metastore chart to use the community MySql chart/image. As such, the aiSSEMBLE-specific image should be deprecated and removed.

DOD

Test Script

  1. Final Test: Verify the 1.9.0-SNAPSHOT tag of the aissemble-hive-service image has been published more recently than the same tag for the aissemble-hive-mysql image
  2. OTS: Verify the module aiSSEMBLE::Extensions::Docker::Hive MySql does not appear in the build summary
  3. Create a project from the current archetype
    mvn archetype:generate -B '-DarchetypeGroupId=com.boozallen.aissemble' \
                          '-DarchetypeArtifactId=foundation-archetype' \
                          '-DarchetypeVersion=1.9.0-SNAPSHOT' \
                          '-DgroupId=org.test' \
                          '-Dpackage=org.test' \
                          '-DprojectGitUrl=test.org/test.git' \
                          '-DprojectName=Test Project' \
                          '-DartifactId=ots-330' \
                          '-DdockerProjectRepositoryUrl=test-registry/' \
    && cd ots-330
  4. Add the old v1 Hive DB profile to the fermenter-mda executions in ots-330-deploy/pom.xml
                       <execution>
                           <id>hive-metastore-db</id>
                           <phase>generate-sources</phase>
                           <goals>
                               <goal>generate-sources</goal>
                           </goals>
                           <configuration>
                               <basePackage>org.test</basePackage>
                               <profile>hive-metastore-db-deploy</profile>
                               <propertyVariables>
                                   <appName>hive-metastore-db</appName>
                               </propertyVariables>
                           </configuration>
                       </execution>
  5. Run the build: mvn clean install
  6. Verify tag: "1.8.1" is specified in ots-330-deploy/src/main/resources/apps/hive-metastore-db/values.yaml
  7. Run the build again to trigger the migrations: mvn clean install
  8. Verify tag: "1.8.1" is still specified in ots-330-deploy/src/main/resources/apps/hive-metastore-db/values.yaml
ewilkins-csi commented 1 month ago

DoD and OTS with @csun-cpointe

carter-cundiff commented 1 month ago

Testing passed: Before migration: image After migration: image