search

ctlab / quant3p

A set of scripts for 3' RNA-seq quantification
Apache License 2.0
1 stars 2 forks source link

quant3p terminates with error #6

Open wbrenne opened 4 years ago

wbrenne commented 4 years ago

After installing quant3p, the test run with the example data fails with an error, for the output see below. As far as I get it, it is missing pybedtools. However, I have installed version 0.8.1 as conda list shows: pybedtools 0.8.1 py37he513fc3_0 bioconda

Are there any ideas / solutions how to get it running? Best, Wolfram

$ quant3p -n example -g example/mm10.slice.gtf example/bam/*.bam Calling peaks... Calling peaks INFO @ Thu, 23 Jan 2020 18:33:44:

Command line: callpeak -t /dev/fd/63 --outdir example.pos.macs -s 26 --nomodel -f BED -g 3e9 -n example.pos -q 0.01

ARGUMENTS LIST:

name = example.pos

format = BED

ChIP-seq file = ['/dev/fd/63']

control file = None

effective genome size = 3.00e+09

band width = 300

model fold = [5, 50]

qvalue cutoff = 1.00e-02

The maximum gap between significant sites is assigned as the read length/tag size.

The minimum length of peaks is assigned as the predicted fragment length "d".

Larger dataset will be scaled towards smaller dataset.

Range for calculating regional lambda is: 10000 bps

Broad region calling is off

Paired-End mode is off

INFO @ Thu, 23 Jan 2020 18:33:44: #1 read tag files... INFO @ Thu, 23 Jan 2020 18:33:44: #1 read treatment tags... INFO @ Thu, 23 Jan 2020 18:33:44:

Command line: callpeak -t /dev/fd/63 --outdir example.neg.macs -s 26 --nomodel -f BED -g 3e9 -n example.neg -q 0.01

ARGUMENTS LIST:

name = example.neg

format = BED

ChIP-seq file = ['/dev/fd/63']

control file = None

effective genome size = 3.00e+09

band width = 300

model fold = [5, 50]

qvalue cutoff = 1.00e-02

The maximum gap between significant sites is assigned as the read length/tag size.

The minimum length of peaks is assigned as the predicted fragment length "d".

Larger dataset will be scaled towards smaller dataset.

Range for calculating regional lambda is: 10000 bps

Broad region calling is off

Paired-End mode is off

INFO @ Thu, 23 Jan 2020 18:33:44: #1 read tag files... INFO @ Thu, 23 Jan 2020 18:33:44: #1 read treatment tags... io.UnsupportedOperation: File or stream is not seekable. Exception ignored in: 'MACS2.IO.Parser.BEDParser.skip_first_commentlines' io.UnsupportedOperation: File or stream is not seekable. io.UnsupportedOperation: File or stream is not seekable. Exception ignored in: 'MACS2.IO.Parser.BEDParser.skip_first_commentlines' io.UnsupportedOperation: File or stream is not seekable. INFO @ Thu, 23 Jan 2020 18:33:44: #1 tag size is determined as 26 bps INFO @ Thu, 23 Jan 2020 18:33:44: #1 tag size = 26.0 INFO @ Thu, 23 Jan 2020 18:33:44: #1 total tags in treatment: 1105 INFO @ Thu, 23 Jan 2020 18:33:44: #1 user defined the maximum tags... INFO @ Thu, 23 Jan 2020 18:33:44: #1 filter out redundant tags at the same location and the same strand by allowing at most 1 tag(s) INFO @ Thu, 23 Jan 2020 18:33:44: #1 tag size is determined as 26 bps INFO @ Thu, 23 Jan 2020 18:33:44: #1 tag size = 26.0 INFO @ Thu, 23 Jan 2020 18:33:44: #1 total tags in treatment: 10956 INFO @ Thu, 23 Jan 2020 18:33:44: #1 user defined the maximum tags... INFO @ Thu, 23 Jan 2020 18:33:44: #1 tags after filtering in treatment: 701 INFO @ Thu, 23 Jan 2020 18:33:44: #1 filter out redundant tags at the same location and the same strand by allowing at most 1 tag(s) INFO @ Thu, 23 Jan 2020 18:33:44: #1 Redundant rate of treatment: 0.37 INFO @ Thu, 23 Jan 2020 18:33:44: #1 finished! INFO @ Thu, 23 Jan 2020 18:33:44: #2 Build Peak Model... INFO @ Thu, 23 Jan 2020 18:33:44: #2 Skipped... INFO @ Thu, 23 Jan 2020 18:33:44: #2 Use 200 as fragment length INFO @ Thu, 23 Jan 2020 18:33:44: #3 Call peaks... INFO @ Thu, 23 Jan 2020 18:33:44: #3 Pre-compute pvalue-qvalue table... INFO @ Thu, 23 Jan 2020 18:33:44: #1 tags after filtering in treatment: 2792 INFO @ Thu, 23 Jan 2020 18:33:44: #1 Redundant rate of treatment: 0.75 INFO @ Thu, 23 Jan 2020 18:33:44: #1 finished! INFO @ Thu, 23 Jan 2020 18:33:44: #2 Build Peak Model... INFO @ Thu, 23 Jan 2020 18:33:44: #2 Skipped... INFO @ Thu, 23 Jan 2020 18:33:44: #2 Use 200 as fragment length INFO @ Thu, 23 Jan 2020 18:33:44: #3 Call peaks... INFO @ Thu, 23 Jan 2020 18:33:44: #3 Pre-compute pvalue-qvalue table... INFO @ Thu, 23 Jan 2020 18:33:44: #3 Call peaks for each chromosome... INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write output xls file... example.neg.macs/example.neg_peaks.xls INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write peak in narrowPeak format file... example.neg.macs/example.neg_peaks.narrowPeak INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write summits bed file... example.neg.macs/example.neg_summits.bed INFO @ Thu, 23 Jan 2020 18:33:44: Done! INFO @ Thu, 23 Jan 2020 18:33:44: #3 Call peaks for each chromosome... INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write output xls file... example.pos.macs/example.pos_peaks.xls INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write peak in narrowPeak format file... example.pos.macs/example.pos_peaks.narrowPeak INFO @ Thu, 23 Jan 2020 18:33:44: #4 Write summits bed file... example.pos.macs/example.pos_summits.bed INFO @ Thu, 23 Jan 2020 18:33:44: Done! Combining peaks 'example.pos.macs/example.pos_summits.bed' wurde entfernt 'example.pos.macs/example.pos_peaks.narrowPeak' wurde entfernt 'example.pos.macs/example.pos_peaks.xls' wurde entfernt Verzeichnis 'example.pos.macs' wurde entfernt 'example.neg.macs/example.neg_peaks.narrowPeak' wurde entfernt 'example.neg.macs/example.neg_peaks.xls' wurde entfernt 'example.neg.macs/example.neg_summits.bed' wurde entfernt Verzeichnis 'example.neg.macs' wurde entfernt Done Extending annotaion... Traceback (most recent call last): File "/home/wb64jozo/.local/bin/gtf-extend", line 6, in from pkg_resources import load_entry_point File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3252, in @_call_aside File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3236, in _call_aside f(*args, kwargs) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3265, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 584, in _build_master ws.require(requires) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 901, in require needed = self.resolve(parse_requirements(requirements)) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 787, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'pybedtools>=0.6.9' distribution was not found and is required by quant3p environment: Zeile 2: htseq-count: Befehl nicht gefunden environment: Zeile 2: htseq-count: Befehl nicht gefunden Traceback (most recent call last): File "/home/wb64jozo/.local/bin/fix-mm", line 6, in from pkg_resources import load_entry_point File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3252, in @_call_aside File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3236, in _call_aside f(*args, *kwargs) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3265, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 584, in _build_master ws.require(requires) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 901, in require needed = self.resolve(parse_requirements(requirements)) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 787, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'pybedtools>=0.6.9' distribution was not found and is required by quant3p Traceback (most recent call last): File "/home/wb64jozo/.local/bin/fix-mm", line 6, in from pkg_resources import load_entry_point File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3252, in @_call_aside File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3236, in _call_aside f(args, kwargs) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3265, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 584, in _build_master ws.require(requires) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 901, in require needed = self.resolve(parse_requirements(requirements)) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 787, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'pybedtools>=0.6.9' distribution was not found and is required by quant3p environment: Zeile 2: htseq-count: Befehl nicht gefunden environment: Zeile 2: htseq-count: Befehl nicht gefunden Traceback (most recent call last): File "/home/wb64jozo/.local/bin/fix-mm", line 6, in from pkg_resources import load_entry_point File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3252, in Traceback (most recent call last): File "/home/wb64jozo/.local/bin/fix-mm", line 6, in from pkg_resources import load_entry_point File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3252, in @_call_aside File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3236, in _call_aside @_call_aside File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3236, in _call_aside f(*args, *kwargs) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3265, in _initialize_master_working_set f(args, **kwargs) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 3265, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 584, in _build_master ws.require(requires) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 901, in require needed = self.resolve(parse_requirements(requirements)) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 787, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'pybedtools>=0.6.9' distribution was not found and is required by quant3p working_set = WorkingSet._build_master() File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 584, in _build_master ws.require(requires) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 901, in require needed = self.resolve(parse_requirements(requirements)) File "/home/wb64jozo/.local/lib/python3.6/site-packages/pkg_resources/init.py", line 787, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'pybedtools>=0.6.9' distribution was not found and is required by quant3p quant3p: Error on or near line 121; exiting with status 1

assaron commented 4 years ago

Did you try to upgrade pybedtools?

jgarces02 commented 4 years ago

Hi. I had the same error and what I've seen is that count files (those obtained with htseq-count) are correctly generated in NAME.htseq.out, the problem seems to appear in the final part of the script (from line 121) where these count files are merged together, so I guess it's not a problem of versions.

This final code seems not to have nothing related to quant3p dependences (I mean: SAMtools, MACS2 or pybedtools)... it's like this: 

find "${BAMS[@]}" -print0 | xargs -0 -n 1 -P "${NPROC}" bash -c 'count_bam "$@"' _ 

if [ -z "${KEEP_TEMP}" ]; then
    rm "${fixed_gtf}"
    rm "${peaks}"
fi

tags=()
htseq_outs=()

header=""
fields="1"
i=0

for bam in "${BAMS[@]}"
do
    tag=`basename "${bam}" .bam`
    htseq_out="${htseq_dir}/${tag}.htseq.out"
    tags+=("${tag}")
    htseq_outs+=("${htseq_out}")
    header="${header}\t${tag}"
    i=$(($i+2))
    fields="${fields},${i}"
done

cat <(echo -e "${header}") \
    <(paste "${htseq_outs[@]}" | cut -f "${fields}") \
    > "${NAME}.cnt"

if [ -z "${KEEP_TEMP}" ]; then
    for htseq_out in "${htseq_outs[@]}"
    do
        rm "${htseq_out}"
    done

    rmdir "${htseq_dir}"
fi

echo "Done"

Or maybe I'm missconsidering something. Thanks anyway for this straightforward script!

akiss-me commented 3 years ago

I solved a similar problem by replacing the conda environment and the corresponding versions of the required software with python2.7.

conda create --name quant3p_env_python2.7 python=2.7 conda activate quant3p_env_python2.7 conda install -c bioconda macs2=2.1.4 pybedtools=0.8.1 pysam=0.16.0.1 HTseq=0.11.3 conda install -c bioconda samtools=1.9 --force-reinstall

akiss-me commented 2 years ago

0.11.3

update:

conda install -y -c bioconda -c defaults \ macs2=2.1.4 \ pybedtools=0.8.1 \ pysam=0.15.3 \ HTseq=0.11.3 \ samtools=1.9