Query on Estimated Running Times Across different Datasets

USM-CHU-FGuyon / BlendedICU

OMOP standardization pipeline for ICU databases

MIT License

26 stars 9 forks source link

Query on Estimated Running Times Across different Datasets #24

Closed xinyuejohn closed 6 months ago

xinyuejohn commented 7 months ago

Could you provide insights or benchmarks regarding the time it takes to run the pipeline with various datasets? Moreover, a progress bar indicating the estimated running time would also be beneficial.

Having an estimate of the running time before initiating the pipeline would be extremely helpful.

Thank you!

USM-CHU-FGuyon commented 7 months ago

Hi, I will definetely add this sometime soon !

USM-CHU-FGuyon commented 6 months ago

I did not add a progressbar (yet), but I added estimated running times on most lengthy scripts. I timed this running them all in parallel, so times could be a little overestimated for a single run. Running times may also depend a lot on writing times on specific machines.

Please tell me if you find consistent running times, I never timed this on another machine.