Create a prod collector -- this is based on the dev collector that exists here after I updated it yesterday/this morning. For now this is essentially a copy/paste of the dev job, but keeping them separate means we can iterate on the dev job for the more generic solution without impacting our production MVP.
This prod job will be triggered a prod airflow job and write data to tables to a prod role in GCP.
testing
walked through the Readme steps and confirmed I could run this locally
Checklist for reviewer:
[X] Commits should reference a bug or github issue, if relevant (if a bug is
referenced, the pull request should include the bug number in the title)
[X] Scan the PR and verify that no changes (particularly to
.circleci/config.yml) will cause environment variables (particularly
credentials) to be exposed in test logs
[X] Ensure the container image will be using permissions granted to
telemetry-airflow
responsibly.
https://mozilla-hub.atlassian.net/browse/AE-364
Create a prod collector -- this is based on the dev collector that exists here after I updated it yesterday/this morning. For now this is essentially a copy/paste of the dev job, but keeping them separate means we can iterate on the dev job for the more generic solution without impacting our production MVP.
This prod job will be triggered a prod airflow job and write data to tables to a prod role in GCP.
testing
walked through the Readme steps and confirmed I could run this locally
Checklist for reviewer:
.circleci/config.yml) will cause environment variables (particularly credentials) to be exposed in test logs