How looper write to divvy template before the submission?

[arpanda]

In my divvy template, I have few variables like like 'JOBNAME' and 'CODE'. I would like to pass the values from looper run. But I could not find the details in the documentation. So can you please help me to configure my looper template?

[nsheff]

It's looper run --compute.

Try looper run --compute JOBNAME=name CODE=ls

http://looper.databio.org/en/latest/variable-namespaces/

[nsheff]

If this isn't what you're looking for, I may not understand your question. Can you paste your divvy configuration file? are you using divvy adapters?

[arpanda]

Thanks for the reply. My explanation was not good. I started with the hello_looper example to learn its working flow. Earlier, I setup the divvy configuration for the SGE cluster. Here is divvy template '

#!/bin/bash
#$ -cwd
#$ -o logs
#$ -N '{JOBNAME}'
#$ -q 1-day
#$ -pe threaded 10
#$ -j y
#$ -l h_vmem=10G
#$ -l h_stack=10M
#$ -V

echo 'Compute node:' `hostname`
echo 'Start time:' `date +'%Y-%m-%d %T'`
echo {sample}

{CODE}

I would like to run the pipeline/count_lines.sh program along with the some other program through the divvy template. But I could not figured it out by myself. Specifically, how the pipeline interface works with the cluster job submission system (Divvy template).

[nsheff]

I'd like to see your divvy config file. Can you show the output of divvy list and cat $DIVCFG ?

[arpanda]

divvy list( Here I trimmed divvy config path) shows

Using divvy config: common_data/divvy_config.yaml
Available compute packages:

default
develop_package
local
big

And cat $DIVCFG shows

compute_packages:
  default:
    submission_template: divvy_templates/sge_template.sub
    submission_command: qsub
  local:
    submission_template: divvy_templates/local_template.sub
    submission_command: sh
  develop_package:
    submission_template: divvy_templates/slurm_template.sub
    submission_command: sbatch
    partition: develop
  big:
    submission_template: divvy_templates/slurm_template.sub
    submission_command: sbatch
    partition: bigmem

[stolarczyk]

I think adapters section is what you're missing in your divvy config. Please read "Mapping variables to submission templates using divvy adapters" section here: http://looper.databio.org/en/latest/variable-namespaces/

[arpanda]

Now pipeline/count_lines.sh working properly for me in SGE cluster. Thanks @nsheff and @stolarczyk for helping me to sort out the procedure.