chip-seq-pipeline2 run with conda on linux with ERROR

[yingsun-ucsd]

I used chip-seq-pipeline from ENCODE and wanted to switch to this new version. Unfortunately, I can not get it run on Linux. I tried to follow the instructions to install all required packages etc. Then I cd into chip-seq-pipeline2, where chip.croo.v5.json and chip.wdl located. I tried to run the test as below:

caper run chip.wdl [CaperURI] copying from url to local, src: https://github.com/broadinstitute/cromwell/releases/download/47/cromwell-47.jar [CaperURI] copying skipped, target: /home/ysun/.caper/cromwell_jar/cromwell-47.jar [CaperURI] copying from url to local, src: https://github.com/broadinstitute/cromwell/releases/download/47/womtool-47.jar [CaperURI] copying skipped, target: /home/ysun/.caper/womtool_jar/womtool-47.jar [Caper] Validating WDL/input JSON with womtool... Required workflow input 'chip.pipeline_type' not specified [Caper] Error (womtool): WDL or input JSON is invalid.

Do you know what is the problem here? I can not find the manual or document to run chip-seq-pipeline2 on local Linux. Would you mind pointing me to that if possible? I followed the instructions for the json input files but have no idea about how, if need, to generate the chip.wdl file. I am sorry if it's a very basic question. I just felt very lost here.

[leepc12]

I think your Caper is outdated. cromwell-47.jar is too old. Please remove cromwell= line from your Caper conf (~/.caper/default.conf). Also check Caper's version. Make sure that it's >=2.0.0.

$ caper -v

Please post your input JSON. chip.pipeline_type is missing?

[yingsun-ucsd]

Thanks. 1) my caper conf looks very blank: more ~/.caper/default.conf

backend=local tmp-dir=/nfs/lab/tmp

2) Yes, my caper is very old (0.6.1), I will update it.

3) I just used the wdl in the package which I believe is this one https://github.com/ENCODE-DCC/chip-seq-pipeline2/blob/master/chip.wdl. I did not specify any json file as input. I don't understand very well the whole process. And I can not find the manual or document to run chip-seq-pipeline2 on local Linux. Would you mind pointing me to that if possible? I followed the instructions for the json input files but have no idea about how, if need, to generate the chip.wdl file. I am sorry if it's a very basic question. I just felt very lost here.

[leepc12]

Please read Caper's README. You need to caper init [YOUR_BACKEND] first. Follow the instruction in README according to your chosen backend.

Use https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI_subsampled_chr19_only.json as an input.

Something like

$ caper run chip.wdl https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI_subsampled_chr19_only.json --docker

Choose a flag for your environment. Choices are --docker, --singularity and --conda. If you use Conda, install Miniconda3 first and install pipeline's Conda environment (scripts/install_conda_env.sh).

[yingsun-ucsd]

I redid it from the very beginning on a new server. And here are the exact steps I did. But I still got an error. Debugging output attached. I changed the name from "cromwell.out" to "cromwell.txt", otherwise, I can not attach.

---

$ pip install pip --upgrade $ export PATH="/home/ysun/.local/bin:$PATH" $ pip install caper $ pip install caper --upgrade $ caper init local $ git clone https://github.com/ENCODE-DCC/chip-seq-pipeline2 $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh $ bash Miniconda3-latest-Linux-x86_64.sh $ bash Miniconda3-latest-Linux-x86_64.sh -u $ cd chip-seq-pipeline2/ $ conda --version conda 4.11.0 $ caper --version 2.1.3 $ java --version openjdk 11.0.13 2021-10-19 OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04) OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04, mixed mode, sharing) $ python3 --version Python 3.9.7 $ bash scripts/uninstall_conda_env.sh $ bash scripts/install_conda_env.sh $ mkidr /home/ysun/chip-seq-pipeline2-genome/hg38|hg19|mm10|mm9 $ bash scripts/download_genome_data.sh hg38|hg19|mm10|mm9 /home/ysun/chip-seq-pipeline2-genome/hg38|hg19|mm10|mm9

---

Until this point, everything works fine. Then I tried the example as: THE ERROR HAPPENED AT THIS POINT $ caper run chip.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI_subsampled_chr19_only.json --conda THE ERROR HAPPENED AT THIS POINT

Please advise on what's going wrong here. Thanks. cromwell.txt

[yingsun-ucsd]

265 with more info