Marketing Analytics Jumpstart consists of an easy, extensible and automated implementation of an end-to-end solution that enables Marketing Technology teams to store, transform, enrich with 1PD and analyze marketing data, and programmatically send predictive events to Google Analytics 4 to support conversion optimization and remarketing campaigns.
remove the need of reading the config.yaml file from the feature-store module, which causes the terraform to always replace the datasets and tables when re-applied.
use for each construct to simplify the repeated resources creation of datasets and tables
add new store procedures that aggregates predictions for all the use cases and take account that the prediction table maybe not existing
How has this been tested?
Tested in integration environment
Please explain how you have tested the new changes.
Due to the change of using foreach construct, the resources id has changed in terraform and when pulling this change into an existing deployment run the following mv command to move the existing resources before doing terraform apply:
terraform state mv 'module.feature_store[0].google_bigquery_dataset.audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["audience_segmentation"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.auto_audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["auto_audience_segmentation"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.churn_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["churn_propensity"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.customer_lifetime_value' 'module.feature_store[0].google_bigquery_dataset.datasets["customer_lifetime_value"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.feature_store' 'module.feature_store[0].google_bigquery_dataset.datasets["feature_store"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.purchase_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["purchase_propensity"]'
terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_predictions"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_vbb"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["gemini_insights"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.audience_segmentation_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["audience_segmentation_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_label' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_lifetime_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_lifetime_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_lookback_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_lookback_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_lifetime_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_lifetime_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_segmentation_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_segmentation_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_segmentation_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_segmentation_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_session_event_aggregated_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_session_event_aggregated_metrics"]'
terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_table.main["latest"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_predictions_latest"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_correlation"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_correlation"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_daily"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_daily"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_weekly"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["vbb_weights"]' 'module.feature_store[0].google_bigquery_table.tables["vbb_weights"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_daily"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_daily"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_monthly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_monthly"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_weekly"]'
Checklist
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have successfully run the E2E tests, and have included the links to the pipeline runs below
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I have updated any relevant documentation to reflect my changes
Description
How has this been tested?
Tested in integration environment
Please explain how you have tested the new changes. Due to the change of using foreach construct, the resources id has changed in terraform and when pulling this change into an existing deployment run the following mv command to move the existing resources before doing terraform apply: terraform state mv 'module.feature_store[0].google_bigquery_dataset.audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["audience_segmentation"]' terraform state mv 'module.feature_store[0].google_bigquery_dataset.auto_audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["auto_audience_segmentation"]' terraform state mv 'module.feature_store[0].google_bigquery_dataset.churn_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["churn_propensity"]' terraform state mv 'module.feature_store[0].google_bigquery_dataset.customer_lifetime_value' 'module.feature_store[0].google_bigquery_dataset.datasets["customer_lifetime_value"]' terraform state mv 'module.feature_store[0].google_bigquery_dataset.feature_store' 'module.feature_store[0].google_bigquery_dataset.datasets["feature_store"]' terraform state mv 'module.feature_store[0].google_bigquery_dataset.purchase_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["purchase_propensity"]' terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_predictions"]' terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_vbb"]' terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["gemini_insights"]' terraform state mv 'module.feature_store[0].google_bigquery_table.audience_segmentation_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["audience_segmentation_inference_preparation"]' terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_inference_preparation"]' terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_label"]' terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_inference_preparation"]' terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_label' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_label"]' terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_inference_preparation"]' terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_label"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_dimensions"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_lifetime_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_lifetime_dimensions"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_lookback_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_lookback_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_lifetime_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_lifetime_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_segmentation_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_segmentation_metrics"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_segmentation_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_segmentation_dimensions"]' terraform state mv 'module.feature_store[0].google_bigquery_table.user_session_event_aggregated_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_session_event_aggregated_metrics"]' terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_table.main["latest"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_predictions_latest"]' terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_correlation"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_correlation"]' terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_daily"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_daily"]' terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_weekly"]' terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["vbb_weights"]' 'module.feature_store[0].google_bigquery_table.tables["vbb_weights"]' terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_daily"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_daily"]' terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_monthly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_monthly"]' terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_weekly"]'
Checklist
Pipeline run links: