Create a test function to transform model-output data

[bsweger]

Following the "demos not memos" principle, before making a decision about what will trigger model-output data conversions for cloud usage (#20), it might be helpful to see some preliminary code for these data conversions.

Create a prototype class.function in hubverse-cloud that:

• [x] accepts filename and S3 location as input
• [x] read file into an in-memory tabular data structure (e.g., R dataframe, pandas dataframe, arrow)
• [x] parse filename into team, model, round (see logic in hubValidations)
• [x] add 3 columns to the tabular data structure: round id, team name, model name
• [x] writes the modified data (with the new columns) to the specified S3 location in parquet format

[bsweger]

For lack of a better place to put it, there is now a hubverse-transforms branch in the infrastructure repo that has Python function for doing the column/format model-output transformations: https://github.com/Infectious-Disease-Modeling-Hubs/hubverse-infrastructure/tree/bsweger/add-model-output-transforms/hubverse-transforms

It's written in Python, which is an out-of-the-box supported AWS Lambda runtime environment (i.e., we'll have an easier path to testing lambda functions triggered by S3 updates, if that's something we want to explore).

[bsweger]

Marking this one closed, since the the goal was getting some prototype code up and running as a proof-of-concept for converting incoming model-output data to parquet format.

Those on Confluence can follow along with some WIP experiments using this function: https://reichlab.atlassian.net/wiki/spaces/RLD/pages/13631576/Automated+model-output+transforms