CMakePP / CMinx

Generates API documentation for CMake functions and macros
https://cmakepp.github.io/CMinx/
Apache License 2.0
14 stars 5 forks source link

Parallelising multiple CMake file processings #15

Closed AutonomicPerfectionist closed 2 years ago

AutonomicPerfectionist commented 4 years ago

Using the Python Multiprocessing library we can generate multiple RST files in parallel by launching a new process for every CMake file (or for every twenty or so files depending on testing results). This could potentially speed up large documentation runs but could also significantly impact smaller runs. Perhaps a check could be used to determine if the batch size is sufficiently large enough? Unfortunately there isn't a platform agnostic way to do parallel operations in Python without additional processes due to the GIL and the fact that Windows does not implement fork(), so there is a significant startup time for every new process.

ryanmrichard commented 4 years ago

I think this would be a nice feature; however, I wouldn't call it a priority. Assuming the command to process file.cmake is something like ./cmakedoc file.cmake, on Linux something like ls *.cmake | xargs - P n -I % ./cmakedoc % will parallelize over n processes. As it's OS native it's going to be hard to beat.

ryanmrichard commented 2 years ago

I'm going to close this. Like I said, I think it would be nice to have, but until there's a demand for it I don't want to waste resources on coding it up.