So, mocking athena locally is something of a lost cause - there's not much prior art here, and setting up docker to handle this is a big engineering effort.
As an alternative, we can do the following, primarily for testing during PRs:
Set up a new cloudformation environment:
With a dedicated S3/athena
ETL in synthea data
API key for calling from Github CI
Permanent profile for accessing BCH AWS from Github (this will require BCH IT intervention)
With this, we could export count tables to disk, and then use those as comparisons from a fresh ETL run to check queries.
dogversioning
commented
1 year ago
So, mocking athena locally is something of a lost cause - there's not much prior art here, and setting up docker to handle this is a big engineering effort.
As an alternative, we can do the following, primarily for testing during PRs:
With this, we could export count tables to disk, and then use those as comparisons from a fresh ETL run to check queries.