Running without KVS

[kostyat]

What I currently have problems with every time is that I have not installed SLURM on my computer yet, and so every time I need to run the inferelator, I need to disable KVS, which requires me to go into the code and change some parameters (and I always forget which, or rather, what to change them to).

I think there needs to be an option that automatically disables multithreading, for people who don't have SLURM and want to run just on one node.

[kostyat]

Furthermore, it looks like every time I run the inferelator in a new window, I have to link python to the kvs directory again. Is there a way to set it up such that I don't have to do that every time I run the inferelator?

[kostyat]

I will just leave this for future reference, because I always forget how to do this.

In order to run the inferelator on a machine that does not have SLURM installed, you have to first run the command

export PYTHONPATH=$PYTHONPATH:~/kvsstcp (where ~/kvsstcp should be replaced with wherever your kvsstcp directory is located)

Then, in bbsr_tfa_python.py, on line 22, rank needs to be set to 0 (instead of int(os.environ['SLURM_PROCID']).

Then everything should run fine with the command:

python ~/kvsstcp/kvsstcp.py --execcmd 'python bsubtilis_bbsr_workflow_runner.py'

(where ~/kvsstcp is replaced by wherever your kvsstcp directory is located).

[nickdeveaux]

Hey @kostyat,

If you want to run a single-process run on a laptop, or any other machine without SLURM installed, you just need to set to environment variables in your shell:

export SLURM_PROCID=0
export SLURM_NTASKS=1

What I could do is create a micro-script that checks if you have KVS, clones it if you don't, guarantees it is up-to-date if you do, and then sets these variables and runs the commands you pasted above

[kostyat]

Great, that works! So I would suggest to anyone reading this to use what Nick said instead of what I said (also what I said didn't end up working, I think some other line needs to be changed too).

Nick, so when would that micro-script be run? Would it be a bash script that the user needs to run manually? Or would it be run from inside the workflow runner? Because currently, to launch the workflow runner, one still needs to run python ~/kvsstcp/kvsstcp.py --execcmd 'python bsubtilis_bbsr_workflow_runner.py', so that means that kvsstcp already needs to be on the machine.

[nickdeveaux]

@kostyat

here's the branch: https://github.com/simonsfoundation/inferelator_ng/pull/52

To run:

./inferelator_runner_slurmless.sh <specific organism runner>.py

Example:

./inferelator_runner_slurmless.sh bsubtilis_bbsr_workflow_runner.py

[nickdeveaux]

@kostyat @dayanne-castro This is the script that will run the inferelator by downloading and setting up KVS on your laptop without you having to do anything: https://github.com/simonsfoundation/inferelator_ng/pull/52

[AaronWatters]

resolved by #52