Currently, panel_ricardian is a pretty weak collection of scratchwork, ready for the parquet data. It's not clear whether the workflow will scale, though.
[ ] Add finalized region-level dataset(s)
[ ] Convert csv to parquet if necessary. Check out illustrative example using PyArrow in Python, which RStudio support.
[ ] Explore remote database storage
[ ] Test scalability
[ ] If code doesn't scale, consider modeling in chunks, or else downsampling to model.
Currently, panel_ricardian is a pretty weak collection of scratchwork, ready for the parquet data. It's not clear whether the workflow will scale, though.