Reproducible-Science-Curriculum / Reproducible-Science-Hackathon-Dec-08-2014

Workshop bringing together individuals interested in developing curriculum, workflows, and tools to strengthen reproducibility in research.
32 stars 3 forks source link

Pitfalls of Parallelization #8

Open dmlond opened 10 years ago

dmlond commented 10 years ago

Data intensive research, such as Next Generation Sequencing, can require extensive parallelization of analysis routines to efficiently process data in a timely fashion. While parallelization provides marked benefits in the speed of analysis, it creates extra challenges when attempting to share research analysis processes with other researchers.

We could develop suggestions and practices for researchers to use in sharing research analyses involving parallelization. This might involve using parameterization to allow other researchers a choice of whether to use parallelization or not, or provide them the ability to scale its parallelization to the infrastructure available to them.