tbenthompson / cppimport

Import C++ files directly from Python!
MIT License
1.18k stars 67 forks source link

Add configuration for setting the degree of compilation parallelism. #51

Open yaoyuannnn opened 3 years ago

yaoyuannnn commented 3 years ago

Maybe the first question is how many cppimport will use by default. Since I'm running it on a machine with a small memory, sometimes I can see the compilation goes out of memory.

tbenthompson commented 3 years ago

Currently, parallelism is off by default. If you turn parallelism on, the number of cores is equal to multiprocessing.cpu_count().

This would be quite easy to modify. I have two proposals for how we could do this:

Some locations in the code that would be useful for this: https://github.com/tbenthompson/cppimport/blob/00c40d9a7f706409156a42ff3a7934e54a8050c1/cppimport/templating.py#L46 https://github.com/tbenthompson/cppimport/blob/00c40d9a7f706409156a42ff3a7934e54a8050c1/cppimport/build_module.py#L57 https://github.com/tbenthompson/cppimport/blob/00c40d9a7f706409156a42ff3a7934e54a8050c1/cppimport/build_module.py#L150

I would be excited to accept a pull request if you have the time to implement this!

tbenthompson commented 3 years ago

Reopened because this is something I want to do eventually.