MRC-CSO-SPHSU / simpaths_parallel

Code for running SimPaths in parallel
0 stars 0 forks source link

UPD: add missing parts #5

Open vkhodygo opened 1 year ago

vkhodygo commented 1 year ago
andrewbaxter439 commented 1 year ago

Add all elements required to conduct multiruns of SimPaths in parallel.

As a brief list to work on:

andrewbaxter439 commented 1 year ago

Add scripts for data conversion

There were two R scripts deleted in 01df37665552708b01fdfd6eedfba4c39bb38fe7 which convert to arrow/parquet if these are any use? One converts Person.csv, BenefitUnit.csv and Household.csv output files to three parquet files. The other joins them to one combined database.

andrewbaxter439 commented 1 year ago

If possible solving #6 might be priority - want to test things in running order so would add tests for the above files/directories after parsing arguments.

vkhodygo commented 1 year ago

There were two R scripts deleted in 01df376 which convert to arrow/parquet if these are any use? One converts Person.csv, BenefitUnit.csv and Household.csv output files to three parquet files. The other joins them to one combined database.

I do have a slightly better version which I'll add once you're done with #4

vkhodygo commented 1 year ago

If possible solving #6 might be priority - want to test things in running order so would add tests for the above files/directories after parsing arguments.

I'll get to it, thanks for letting me know.