My testing on the error cases was done without cloud auth so I didn't validate the writes to bigquery succeeded. The code didn't enforce any type constraints so I had passed the wrong kind of value to one of the method parameters.
The docker builds were updated and then the dev job ran but failed due to the error TypeError: Object of type datetime is not JSON serializable.
This run was expected to get an error record written to bigquery and not an airflow error since the airflow schedule hasn't switched to only pass in dates at midnight yet.
solution
I reproduced this locally using the same parameters that the airflow job ran with, and then figured out that the base report expected an int for the slot_start field.
ran this before and after the code adjustment. before the change it had the not JSON serializable error, after it succeeded and I confirmed the report result with the error message showed up in bigquery.
Checklist for reviewer:
[X] Commits should reference a bug or github issue, if relevant (if a bug is
referenced, the pull request should include the bug number in the title)
[X] Scan the PR and verify that no changes (particularly to
.circleci/config.yml) will cause environment variables (particularly
credentials) to be exposed in test logs
[X] Ensure the container image will be using permissions granted to
telemetry-airflow
responsibly.
https://mozilla-hub.atlassian.net/browse/AE-457
problem
My testing on the error cases was done without cloud auth so I didn't validate the writes to bigquery succeeded. The code didn't enforce any type constraints so I had passed the wrong kind of value to one of the method parameters.
The docker builds were updated and then the dev job ran but failed due to the error
TypeError: Object of type datetime is not JSON serializable.This run was expected to get an error record written to bigquery and not an airflow error since the airflow schedule hasn't switched to only pass in dates at midnight yet.
solution
I reproduced this locally using the same parameters that the airflow job ran with, and then figured out that the base report expected an int for the slot_start field.
testing
ran this before and after the code adjustment. before the change it had the
not JSON serializableerror, after it succeeded and I confirmed the report result with the error message showed up in bigquery.Checklist for reviewer:
.circleci/config.yml) will cause environment variables (particularly credentials) to be exposed in test logs