hubverse-org / hubverse-transform

Data transform functions for hubverse model-output files
MIT License
1 stars 0 forks source link

Create a way to re-trigger the hubverse-transform lambda for existing model-output S3 files #27

Closed bsweger closed 3 months ago

bsweger commented 3 months ago

Background

It would be good to have a way to force existing "raw" model-output files in a cloud-enabled hub's S3 bucket to trigger the transform AWS lambda (the one that normally runs when new uploads to S3 occur).

A good example of a use case is being able to re-process the 2023-24 CDC FluSight model-output files now that we've deployed a hubverse-transform patch to fix the schema inconsistencies.

This is a "break glass in case of emergency" function for use on Hubverse-hosted hubs, so it doesn't need to be polished/user-facing (in fact, it won't work unless the user has write access to S3 buckets in the Hubverse AWS account).

Definition of done