interactive parallel maestro on multiprocessor machine

[crkrenn]

I have a workflow that tries to run a serial code multiple times interactively on a machine with multiple CPU's.

What I hoped would happen was that the serial code would run in parallel on multiple processors.

What happens is that all of the serial run steps are run sequentially.

Can this be fixed?

(study file below)

Thanks!

-Chris

Note: parallel "code.py -var1 {}" ::: 1 2 3 does run code.py in parallel (https://www.gnu.org/software/parallel/). I have a workflow that tries to run a serial code multiple times interactively on a machine with multiple CPU's.

What I hoped would happen was that the serial code would run in parallel on multiple processors.

What happens is that all of the serial run steps are run sequentially.

Can this be fixed?

(study file below)

Thanks!

-Chris

Note: parallel "code.py -var1 {}" ::: 1 2 3 does run code.py in parallel (https://www.gnu.org/software/parallel/).

description:
# NO batch section
env: 
study:
  - name: run
     description: run
     run:
         cmd: |
            nohup code.py -var1 ($VAR1)

  - name: process
{..}

# model is defined in pgen.py

[FrankD412]

@crkrenn -- I'm confused. Are you trying to parallelize in code.py or are you wanting Maestro to spin up multiple processes using code.py? Your study implies the former, I just wanted to get clarification. Thanks!

[crkrenn]

Hello Frank,

I’d like multiple instances of code.py to run in parallel in different directories.

-Chris

Chris Krenn Lawrence Livermore National Laboratory Krenn1@llnl.govmailto:Krenn1@llnl.gov 925-423-8636

From: Francesco Di Natale notifications@github.com Reply-To: LLNL/maestrowf reply@reply.github.com Date: Tuesday, February 4, 2020 at 7:44 PM To: LLNL/maestrowf maestrowf@noreply.github.com Cc: Ckl krenn1@llnl.gov, Mention mention@noreply.github.com Subject: Re: [LLNL/maestrowf] interactive parallel maestro on multiprocessor machine (#221)

@crkrennhttps://github.com/crkrenn -- I'm confused. Are you trying to parallelize in code.py or are you wanting Maestro to spin up multiple processes using code.py? Your study implies the former, I just wanted to get clarification. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/LLNL/maestrowf/issues/221?email_source=notifications&email_token=ABOJ5VZNYF6UTZOCYUK6CGDRBIYYZA5CNFSM4KP7SE4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK2BI7I#issuecomment-582227069, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABOJ5VYM2LV5BHAABHBFT2LRBIYYZANCNFSM4KP7SE4A.

[FrankD412]

Got it -- so you'd like Maestro to launch multiple processes at once rather than having local execution continue in sequential order. I'll scope something out.