search

inaos / iron-array

2 stars 0 forks source link

Add support for cgroups #596

Open FrancescAlted opened 2 years ago

FrancescAlted commented 2 years ago

During my Blosc2 & ironArray tutorial to Janelia/HMMI, some guy wanted to know it it could be possible to add support for cgroups. I suppose the idea is to only make use of the resources (specially, number of cores) that cgroups gives access to.

For example, if a process is limited to use just 4 cores out of 16 in the system, it makes little sense to set nthreads=16.

FrancescAlted commented 2 years ago

@stoni do you think this makes sense?

stoni commented 2 years ago

Its an interesting question, I'm wondering if its even necessary. Do you mean that we should query the number of cores that we're allowed to use and return an error if the user wants to use more?

FrancescAlted commented 2 years ago

Yes. So in automatic mode (the default), use the allowed number of cores, rather than the physical ones. In manual mode, where the user wants to specify the number of threads, perhaps it is better to not throw any error and let her shoot in her foot (after all, we are all consenting adults). Makes sense?

stoni commented 2 years ago

I would not implement anything yet - if they give us a contract and still want it - fair enough.. but otherwise lets discuss it in more detail

FrancescAlted commented 2 years ago

Fair enough. This is mostly a reminder because I think it would be a good feature to have.