Adding manual commands to the job

[ScheiklP]

In my setup, I need to execute some bash commands before actually starting the training. For example sourcing the bashrc, activating the conda environment, etc. Is there a way to tell the job "please run this sh script, before running python3 script.py"?

[nicolas-schreiber]

In the hydra->launcher dict you could add a list called setup with the bash commands for fully setting up your environment.

For reference: https://hydra.cc/docs/plugins/submitit_launcher/ https://github.com/facebookincubator/submitit/blob/83898f0a3d9d20a2d5efbc215a3e4d708701f3c4/submitit/slurm/slurm.py#L498C15-L498C15

[ScheiklP]

So that would be something like:

hydra:
   launcher:
      setup:
         - "source $HOME/.bashrc"
         - "conda activate sofa"

?

[nicolas-schreiber]

Yes, exactly, I have not really tried it yet, but the documentation and the code seem to confirm it

[balazsgyenes]

Yes exactly. The lines don't even need to be quoted.

There is a branch in development with an example of this usage actually, but even so, the documentation is in submitit.