search

PaulHancock / Aegean

The Aegean source finding program and associated tools
http://aegeantools.rtfd.io/
Other
47 stars 14 forks source link

aegean --priorized doesn't respect --cores option #96

Closed PaulHancock closed 4 years ago

PaulHancock commented 5 years ago

When running in --priorized mode aegean doesn't respect the --cores option. for exaple:

> aegean tests/test_files/1904-66_SIN.fits --table dlme.fits --island --priorized 1 --cores 2 --input dlme_comp.fits --noregroup 
aegean:INFO This is Aegean 2.0.2-(2018-07-19)
aegean:INFO Using 2 cores
aegean:INFO Priorized fitting of sources in input catalog.
aegean:INFO Stage = 1

but looking at the system monitor shows that all 8 of my cores are being used for the priorized fitting.

PaulHancock commented 5 years ago

Bonus! It appears that regular fitting also has this issue...

or rather the issue is that setting a limit on the number of workers in the queue doesn't automatically mean we have only that many cores in use. Some testing has shown to me that even when only two instances of the fitting are being run at any one time, all of my available compute resources will be consumed.

Maybe this is a problem with the pprocess module (which I plan to discontinue using due to lack of support into python 3).

PaulHancock commented 4 years ago

No longer an issue as of 2/3/2020. Not entirely sure what fixed this tbh.