Open valassi opened 6 days ago
Yes gridpack are running on a single core by design (and the code is optimised based on that feature)
Thanks Olivier!
Replying on some of the discussion on skype (about my 'should it?' question): we could rethink this and submit multi core generation.
IMO anyway it also depends on the experiments... 4 single core jobs (even against a shared GPU) are in principle not less efficient than 1 4-core job (provided you really launch 4 jobs). For CMS I understand that CMS typically used to have GEN-SIM jobs, where the SIM multicore part was waiting a on a single core GEN part (then I am not sure if this changed, maybe splitting GEN and SIM), so there the inefficiency would in any case be that a multicore slot is used with (waht is known to be) a single core executable
rethink the strategy? yes we should. Change the strategy? that's we need to think about...
For the moment, CMS is using (afaik) a read-only gridpack such that it launch (typically 8) gridpack executable in parralel within the same job allocation (which is asking for 8 core).
With our current work we do have multiple way to move forward from that situation:
I think it may be interesting to re-think is that e.g. in an HPC environment you get allocated a N GPUs and IIUC those are solely available to your job, one could then of course start N*M gridpacks within the same job (pilot) submission but this may then become difficult for the data management afterwards ...
Question for Olivier: does run.sh support more than one CPU core? Can it? Should it?
Hi @oliviermattelaer just a question for you. This is from bits and pieces of skype with @choij1589 @Saptaparna @roiser
I am trying to understand if run.sh uses more than one core. This does not seem to be the case. The gridpack I have (slightly modified with profiling, but still) has
In other words, I have the impression that
self.cluster_mode = 0
hardcodes the use of only one core?Can you confirm (or otherwise clarify) please? Thanks