MPI or multiprossessing for RMclean3d

CIRADA-Tools / RM-Tools

RM-synthesis, RM-clean and QU-fitting on polarised radio spectra

MIT License

45 stars 23 forks source link

MPI or multiprossessing for RMclean3d #149

Open whjing opened 4 days ago

whjing commented 4 days ago

I am trying to run rmclean with --mpi or --ncores. But it seems none of them work. When I run it with --mpi:

After it has been finished, it looks it still be running ... 47451731936848_ pic

And when I try to use --ncores, the cpu% is less than 100% and it is not speeded up: 47481731936967_ pic

So how could I make use of the multitasking? Is there a more detailed usage? Many thanks!

Cameron-Van-Eck commented 3 days ago

Hi Wenhui:

I haven't looked at the multiprocessing or MPI functionality in a long time -- it's entirely possible that it has broken at some point without me noticing. Unless someone has used it or looked at it more recently (@AlecThomson ?), I don't have any advice right now.

I'm very busy with POSSUM pipeline development right now, but I may have time to investigate and fix this in early December.

Cheers, Cameron

whjing commented 2 days ago

Hi Wenhui:

I haven't looked at the multiprocessing or MPI functionality in a long time -- it's entirely possible that it has broken at some point without me noticing. Unless someone has used it or looked at it more recently (@AlecThomson ?), I don't have any advice right now.

I'm very busy with POSSUM pipeline development right now, but I may have time to investigate and fix this in early December.

Cheers, Cameron

It's not that urgent for me. And thank you for helping!