search

ENCODE-DCC / chip-seq-pipeline2

ENCODE ChIP-seq pipeline
MIT License
241 stars 123 forks source link

Error when organizing output from the function call "count_signal_track_pooled" using croo #125

Closed hoholee closed 4 years ago

hoholee commented 4 years ago

Describe the bug When I ran croo to organize the output from a successfully finished chip-seq pipeline, I got "TypeError" when it tried to gather results from the call-count_signal_track_pooled folder (see error log below)

OS/Platform

Caper configuration file backend=local tmp-dir=/scratch

Error log

[CaperURI] copying done, target: /cndd2/junhao/dnmt3aKO/chipseq/run_encode_chipseq_pipeline2/organized_results_FC_Control_MTF2/signal/rep1/FC_Control_MTF2_rep1_R1.merged.nodup.no_chrM.negative.bigwig
[CaperURI] copying from local to local, src: /cndd2/junhao/dnmt3aKO/chipseq/run_encode_chipseq_pipeline2/chip/ae1cef94-09f3-415c-91e0-d2abf9bd51ce/call-count_signal_track/shard-1/execution/glob-f8fdf5221d9fbe0d1140ab1532779443/FC_Control_MTF2_rep2_R1.merged.nodup.no_chrM.negative.bigwig
[CaperURI] copying done, target: /cndd2/junhao/dnmt3aKO/chipseq/run_encode_chipseq_pipeline2/organized_results_FC_Control_MTF2/signal/rep2/FC_Control_MTF2_rep2_R1.merged.nodup.no_chrM.negative.bigwig
[CaperURI] copying from local to local, src: /cndd2/junhao/dnmt3aKO/chipseq/run_encode_chipseq_pipeline2/chip/ae1cef94-09f3-415c-91e0-d2abf9bd51ce/call-count_signal_track_pooled/execution/glob-079074fef208d29e859788ed8db804b6/basename_prefix.pooled.positive.bigwig
[CaperURI] copying done, target: /cndd2/junhao/dnmt3aKO/chipseq/run_encode_chipseq_pipeline2/organized_results_FC_Control_MTF2/signal/pooled-rep/basename_prefix.pooled.positive.bigwig
Traceback (most recent call last):
  File "/cndd/junhao/anaconda3/envs/encode-chip-seq-pipeline/bin/croo", line 13, in <module>
    main()
  File "/cndd/junhao/anaconda3/envs/encode-chip-seq-pipeline/lib/python3.7/site-packages/croo/croo.py", line 306, in main
    co.organize_output()
  File "/cndd/junhao/anaconda3/envs/encode-chip-seq-pipeline/lib/python3.7/site-packages/croo/croo.py", line 182, in organize_output
    subgraph, full_path, shard_idx)
  File "/cndd/junhao/anaconda3/envs/encode-chip-seq-pipeline/lib/python3.7/site-packages/croo/croo.py", line 237, in __interpret_inline_exp
    result = result.replace(m.group(0), str(eval(m.group(1))), 1)
  File "<string>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

solution

I believe this is because that line 346 and line 354 in the file chip.croo.v3.json are swapped: https://github.com/ENCODE-DCC/chip-seq-pipeline2/blob/209a71d1041a6e185c8ec9c113ebbeffc6c43abd/chip.croo.v3.json#L335-L362

Simply swap these two lines will resolve the issue:

  "chip.count_signal_track": {
    "pos_bw": {
      "path": "signal/rep${i+1}/${basename}",
      "table": "Signal/Replicate ${i+1}/Count signal track (positive)",
      "node": "[shape=box style=\"filled, rounded\" fillcolor=lightyellow label=\"Count +\\nSignal\"]",
      "subgraph": "cluster_rep${i+1}"
    },
    "neg_bw": {
      "path": "signal/rep${i+1}/${basename}",
      "table": "Signal/Replicate ${i+1}/Count signal track (negative)",
      "node": "[shape=box style=\"filled, rounded\" fillcolor=lightyellow label=\"Count -\\nSignal\"]",
      "subgraph": "cluster_rep${i+1}"
    }
  },
  "chip.count_signal_track_pooled": {
    "pos_bw": {
      "path": "signal/pooled-rep/${basename}",
      "table": "Signal/Pooled replicate/Count signal track (positive)",
      "node": "[shape=box style=\"filled, rounded\" fillcolor=lightyellow label=\"Count +\\nSignal\"]",
      "subgraph": "cluster_pooled_rep"
    },
    "neg_bw": {
      "path": "signal/pooled-rep/${basename}",
      "table": "Signal/Pooled replicate/Count signal track (negative)",
      "node": "[shape=box style=\"filled, rounded\" fillcolor=lightyellow label=\"Count -\\nSignal\"]",
      "subgraph": "cluster_pooled_rep"
    }
  },
leepc12 commented 4 years ago

I just made a fix commit e52cd88cc87b7600e0f0c381136ea05854e16bdb and also uploaded the v4 JSON.

This will be added to the next release but you can still use it by defining it in Croo's command line

$ croo ... --out-def-json https://storage.googleapis.com/encode-pipeline-output-definition/chip.croo.v4.json
hoholee commented 4 years ago

Cool! I'm closing this issue~