Booz Allen's lean manufacturing approach for holistically designing, developing and fielding AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure operational deployment
Other
34
stars
8
forks
source link
Feature: integrate Universal Config store in the hive metastore service #484
Given that aiSSEMBLE support Universal config store service, It is great to make use of this service for consumer in order to prevent duplication in configuration.
First step is to try with one service called Hive Metastore service.
DOD
[ ] Enable overriding hive metastoreServiceConfig.baseProperties in metastoreServiceConfig.properties
[ ] Replace Hive Username and password with Universal config store.
Add the SparkPipeline.json file to the test-hive1-pipeline-models/src/main/resources/pipelines directory
Add to the fermenter-mda plugin executions in test-hive1-deploy/pom.xml
<execution>
<id>configuration-store</id>
<phase>generate-sources</phase>
<goals>
<goal>generate-sources</goal>
</goals>
<configuration>
<basePackage>com.boozallen.aissemble.test</basePackage>
<profile>configuration-store-deploy-v2</profile>
<!-- The property variables below are passed to the Generation Context and utilized
to customize the deployment artifacts. -->
<propertyVariables>
<appName>configuration-store</appName>
</propertyVariables>
</configuration>
</execution>
Run mvn clean install until all the manual actions are complete
Once the manual actions are complete, run mvn clean install -Dmaven.build.cache.skipCache=true once to get any remaining manual actions
add this to the tiltfile
# For WSL users, the configuration files need to be in an accessible path. Update the project path to the root file system. Example: '/mnt/c' or '/mnt/wsl/rancher-desktop'
project_path = os.path.abspath('.')
# Update configuration_files_path to match the path of the config files to be loaded into the configuration store. Example 'my-project-deploy/src/main/resources/config'
configuration_files_path = 'src/main/resources/configurations'
- In the Rancher Desktop, Go to Cluster Dashboard, click on storage-> configMaps to verify configuration is fetching from config service.
-Verify in the webhook log that config injection is successful.
## References/Additional Context
Helm Unit Test (https://github.com/helm-unittest/helm-unittest?tab=readme-ov-file#dependent-subchart-testing)
Description
Given that aiSSEMBLE support Universal config store service, It is great to make use of this service for consumer in order to prevent duplication in configuration. First step is to try with one service called Hive Metastore service.
DOD
Stretch Goal
Test Strategy/Script
mvn clean install -Dmaven.build.cache.skipCache=true
once to get any remaining manual actionsload('ext://helm_resource', 'helm_resource') helm_resource( name='configuration-store', release_name='configuration-store', chart='test-hive1-deploy/src/main/resources/apps/configuration-store', namespace='config-store-ns', flags=['--values=test-hive1-deploy/src/main/resources/apps/configuration-store/values.yaml', '--values=test-hive1-deploy/src/main/resources/apps/configuration-store/values-dev.yaml', '--set=aissemble-configuration-store-chart.configurationVolume.volumePathOnNode=' + project_path + '/' + configuration_files_path, '--create-namespace'] )