A complete reimplementation of the CDM processing, using arrow and DuckDB processing of entire partitions at once, instead of using mainly pandas processing one patient at a time. Perhaps harder to read the code, but it runs approximately 30 times faster.
A complete reimplementation of the CDM processing, using arrow and DuckDB processing of entire partitions at once, instead of using mainly pandas processing one patient at a time. Perhaps harder to read the code, but it runs approximately 30 times faster.