t-neumann / slamdunk

Streamlining SLAM-seq analysis with ultra-high sensitivity
GNU Affero General Public License v3.0
38 stars 23 forks source link

run error #25

Closed xbing88 closed 6 years ago

xbing88 commented 6 years ago

Please help:

-mbp:Slam-seq xbing88$ slamdunk filter -o output -b mm10_miRNA.bed S13_bowtie2_sorted.bam Running slamDunk filter for 1 files (1 threads) [W::hts_idx_load2] The index file is older than the data file: S13_bowtie2_sorted.bam.bai Traceback (most recent call last): File "/Users/xbing88/miniconda2/bin/slamdunk", line 11, in sys.exit(run()) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 450, in run results = Parallel(n_jobs=n, verbose=verbose)(delayed(runFilter)(tid, args.bam[tid], args.bed, args.mq, args.identity, args.nm, outputDirectory) for tid in range(0, len(args.bam))) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 779, in call while self.dispatch_one_batch(iterator): File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 625, in dispatch_one_batch self._dispatch(tasks) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 588, in _dispatch job = self._backend.apply_async(batch, callback=cb) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/joblib/_parallel_backends.py", line 111, in apply_async result = ImmediateResult(func) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/_parallel_backends.py", line 332, in init self.results = batch() File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 131, in call return [func(*args, **kwargs) for func, args, kwargs in self.items] File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 170, in runFilter filter.Filter(bam, outputBAM, getLogFile(outputLOG), bed, mq, minIdentity, maxNM, printOnly, verbose) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/slamdunk/dunks/filter.py", line 268, in Filter mappedReads, unmappedReads, filteredReads, mqFiltered, idFiltered, nmFiltered, multimapper = multimapUTRRetainment (infile, outfile, bed, minIdentity, NM, log) File "/Users/xbing88/miniconda2/lib/python2.7/site-packages/slamdunk/dunks/filter.py", line 105, in multimapUTRRetainment if(float(read.get_tag("XI")) < minIdentity): File "pysam/libcalignedsegment.pyx", line 2383, in pysam.libcalignedsegment.AlignedSegment.get_tag (pysam/libcalignedsegment.c:27819) File "pysam/libcalignedsegment.pyx", line 2425, in pysam.libcalignedsegment.AlignedSegment.get_tag (pysam/libcalignedsegment.c:27229) KeyError: "tag 'XI' not present"

ChiWPak commented 6 years ago

Start here - The index file is older than the data file: S13_bowtie2_sorted.bam.bai. Re-index this bam file

xbing88 commented 6 years ago

Thanks for the comment but this is not the reason it is not working.

t-neumann commented 6 years ago

Hi @xbing88,

did I get this right that you did the mapping with bowtie2?

That's not how you are supposed to use slamdunk, you'll not make any use of the nt-conversion aware alignment and will be missing essential bam-tags (as the said XI).

xbing88 commented 6 years ago

@t-neumann Hey yes thank you this is definitely the problem. Can I install just ngm and remap the reads with that and plug into slamdunk starting at filter? I am having trouble installing slamdunk on our campus cluster.

t-neumann commented 6 years ago

Can you tell me what is causing you troubles? I could be quite tedious to run it from a plain NGM installation.

xbing88 commented 6 years ago

This happens after a few hundred lines into the installation, looks like it thinks Cython is not installed but it is:

copying pybedtools/_Window.c -> build/lib.linux-x86_64-2.7/pybedtools running build_ext building 'pybedtools.cbedtools' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/src creating build/temp.linux-x86_64-2.7/pybedtools gcc -pthread -B /home/xb37w/miniconda2/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isrc/ -I/home/xb37w/miniconda2/include/python2.7 -c src/fileType.cpp -o build/temp.linux-x86_64-2.7/src/fileType.o gcc: error trying to exec 'cc1plus': execvp: No such file or directory error: command 'gcc' failed with exit status 1


Failed building wheel for pybedtools Running setup.py clean for pybedtools Failed to build slamdunk pybedtools Installing collected packages: pybedtools, sortedcontainers, intervaltree, pytz, numpy, python-dateutil, pandas, biopython, Cython, slamdunk Running setup.py install for pybedtools ... error Complete output from command /home/xb37w/miniconda2/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-4aFtCh/pybedtools/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-D2O3Mi-record/install-record.txt --single-version-externally-managed --compile --user --prefix=: running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/genome_registry.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/parallel.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/version.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/main.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/logger.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/stats.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/init.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/helpers.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/filenames.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/bedtool.py -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/settings.py -> build/lib.linux-x86_64-2.7/pybedtools creating build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/regression_tests.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_contrib.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/tfuncs.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_helpers.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_gzip_support.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test1.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_cbedtools.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_iter.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/init.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_len_leak.py -> build/lib.linux-x86_64-2.7/pybedtools/test copying pybedtools/test/test_scripts.py -> build/lib.linux-x86_64-2.7/pybedtools/test creating build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/bigbed.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/plotting.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/init.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/classifier.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/long_range_interaction.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/venn_maker.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/bigwig.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib copying pybedtools/contrib/intersection_matrix.py -> build/lib.linux-x86_64-2.7/pybedtools/contrib creating build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/peak_pie.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/venn_gchart.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/venn_mpl.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/init.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/annotate.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/py_ms_example.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/intron_exon_reads.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts copying pybedtools/scripts/intersection_matrix.py -> build/lib.linux-x86_64-2.7/pybedtools/scripts creating build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/init.py -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/snps.bed.gz -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/vcf-stderr-test.vcf -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/b.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/bedpe.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/rmsk.hg18.chr21.small.bed.gz -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/test_bedpe.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/test.fa.fai -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.gff.gz -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/expand_test.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/test_peaks.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/Cp190_Mbn2_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/multibamcov_test.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/x.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/x.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/a.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/a.igv_script -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.50.200.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/bedpe2.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/m1.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/venn.b.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/SuHw_Kc_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/dm3-chr2L-5M.gff.gz -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.1.100.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/y.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/a.links.html -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.sorted.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/rmsk.hg18.chr21.small.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.sorted.bam.bai -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/hg19.gff -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/Cp190_Kc_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/d.gff -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/mm9.bed12 -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/BEAF_Kc_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/democonfig.yaml -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/dm3-chr2L-5M-invalid.gff.gz -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.50.200.bam.bai -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/hg38-base.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/vcf-stderr-test.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/tag_test2.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/CTCF_Kc_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/BEAF_Mbn2_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/small.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/CTCF_Mbn2_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/SuHw_Mbn2_Bushey_2009.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/c.gff -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/hg38-problem.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.othersort.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/tag_test1.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/venn.c.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/reads.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/164.gtf -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/issue_121.bam -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/small.fastq -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/1000genomes-example.vcf -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/test_tsses.bed -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.1.100.bam.bai -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/v.vcf -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/gdc.gff -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/exons.gff -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/test/data/test.fa -> build/lib.linux-x86_64-2.7/pybedtools/test/data copying pybedtools/_Window.pyx -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/featurefuncs.pyx -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/cbedtools.pyx -> build/lib.linux-x86_64-2.7/pybedtools copying pybedtools/cbedtools.pxd -> build/lib.linux-x86_64-2.7/pybedtools running build_ext Traceback (most recent call last): File "", line 1, in File "/tmp/pip-build-4aFtCh/pybedtools/setup.py", line 385, in setuptools_kwargs) File "/home/xb37w/miniconda2/lib/python2.7/site-packages/setuptools/init.py", line 129, in setup return distutils.core.setup(attrs) File "/home/xb37w/miniconda2/lib/python2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/home/xb37w/miniconda2/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/home/xb37w/miniconda2/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/xb37w/miniconda2/lib/python2.7/site-packages/setuptools/command/install.py", line 61, in run return orig.install.run(self) File "/home/xb37w/miniconda2/lib/python2.7/distutils/command/install.py", line 563, in run self.run_command('build') File "/home/xb37w/miniconda2/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/xb37w/miniconda2/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/xb37w/miniconda2/lib/python2.7/distutils/command/build.py", line 127, in run self.run_command(cmd_name) File "/home/xb37w/miniconda2/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/xb37w/miniconda2/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/xb37w/miniconda2/lib/python2.7/distutils/command/build_ext.py", line 340, in run self.build_extensions() File "/tmp/pip-build-4aFtCh/pybedtools/setup.py", line 269, in build_extensions self.check_cython_extensions(self.extensions) File "/tmp/pip-build-4aFtCh/pybedtools/setup.py", line 266, in check_cython_extensions """ % src) Exception: Cython-generated file 'pybedtools/cbedtools.cxx' not found. Cython is required to compile pybedtools from a development branch. Please install Cython or download a source release of pybedtools.

----------------------------------------

Command "/home/xb37w/miniconda2/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-4aFtCh/pybedtools/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-D2O3Mi-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-4aFtCh/pybedtools/

t-neumann commented 6 years ago

What does this give you?

pip freeze | grep Cython

Is Docker/Singularity an alternative you could use on the cluster? This would be the ideal workaround otherwise.

xbing88 commented 6 years ago

@t-neumann Nothing happens when I do that. Same error msg from installer. I will try to see if I can use Docker. Thank you so much for the help!

t-neumann commented 6 years ago

Well but then this means Cython is not installed.

We use Singularity (which bootstraps Docker images) on our Cluster and it's a supernice workaround. Should deal with any of the above issues.

xbing88 commented 6 years ago

While I'm trying to get Docker to work, I ran ngm for my fastq file and ran slamdunk locally from filter, worked like a charm until "count", where the error for tag came up again. Is there a fix for this:

xinyangs-mbp:Slam-seq xinyangbing$ slamdunk count -o ./data/ -s ./data/ -r miRNA.fa -b mm10_miRNA.bed ./data/S41_filtered.bam Running slamDunk tcount for 1 files (1 threads) Traceback (most recent call last): File "/Users/xinyangbing/miniconda2/bin/slamdunk", line 11, in sys.exit(run()) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 474, in run results = Parallel(n_jobs=n, verbose=verbose)(delayed(runCount)(tid, args.bam[tid], args.ref, args.bed, args.maxLength, args.minQual, args.strictTCs, outputDirectory, snpDirectory) for tid in range(0, len(args.bam))) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 779, in call while self.dispatch_one_batch(iterator): File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 625, in dispatch_one_batch self._dispatch(tasks) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 588, in _dispatch job = self._backend.apply_async(batch, callback=cb) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/_parallel_backends.py", line 111, in apply_async result = ImmediateResult(func) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/_parallel_backends.py", line 332, in init self.results = batch() File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/joblib/parallel.py", line 131, in call return [func(*args, **kwargs) for func, args, kwargs in self.items] File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 190, in runCount maxLength = estimateMaxReadLength(bam) File "/Users/xinyangbing/miniconda2/lib/python2.7/site-packages/slamdunk/utils/misc.py", line 99, in estimateMaxReadLength minLength = min(minLength, read.query_length + read.get_tag("XA")) File "pysam/libcalignedsegment.pyx", line 2383, in pysam.libcalignedsegment.AlignedSegment.get_tag (pysam/libcalignedsegment.c:27819) File "pysam/libcalignedsegment.pyx", line 2425, in pysam.libcalignedsegment.AlignedSegment.get_tag (pysam/libcalignedsegment.c:27229) KeyError: "tag 'XA' not present"

t-neumann commented 6 years ago

Hi,

yeah as I said it's quite tedious:

Did you check whether you checked out the correct ngm version? (0.5.2)

And then you also need to make sure to run it with the same parameter set, which should be something like this (assuming you used QuantSeq and use multimapper recovery):

'ngm -b -r reference.fasta -q fastq-file -t threads --slam-seq 2 -5 12 -l --rg-id readgroup --rg-sm samplename:sampletype:sampletime -n 100 --strata -o out.bam'

I really recommend to run it out of the box to not run into mistakes.

sharonswu commented 6 years ago

Hi, this may not be what's causing the error and please correct me if I'm wrong since I'm not familiar with C++, but looking through the source code for NextGenMap, it seems like --slam-seq 2 would evaluate to False based on line 57 in NextGenMap/src/CS.cpp: static bool const slamSeq = Config.GetInt(SLAM_SEQ) & 0x4; If that's the case, perhaps there are SLAM-seq specific tags that NGM is supposed to be (and is assumed to be) adding but actually is not due to incorrect invocation?

xbing88 commented 6 years ago

I am not trying to run slamdunk on singularity on our cluster and I get this error, please help (loading updated numpy installed on the cluster does not help):

[xb37w@ghpcc06 ~]$ singularity shell docker://tobneu/slamdunk Docker image path: index.docker.io/tobneu/slamdunk:latest Cache folder set to /home/xb37w/.singularity/docker Creating container runtime... Exploding layer: sha256:50aff78429b146489e8a6cb9334d93a6d81d5de2edc4fbf5e2d4d9253625753e.tar.gz Exploding layer: sha256:f6d82e297bce031a3de1fa8c1587535e34579abce09a61e37f5a225a8667422f.tar.gz Exploding layer: sha256:275abb2c8a6f1ce8e67a388a11f3cc014e98b36ff993a6ed1cc7cd6ecb4dd61b.tar.gz Exploding layer: sha256:9f15a39356d6fc1df0a77012bf1aa2150b683e46be39d1c51bc7a320f913e322.tar.gz Exploding layer: sha256:fc0342a94c89e477c821328ccb542e6fb86ce4ef4ebbf1098e85669e051ef0dd.tar.gz Exploding layer: sha256:96006874cf3fecdc2a5294f0069888f750ed15de87a449acd0b7d6a910d20568.tar.gz Exploding layer: sha256:c6a9ef4b9995d615851d7786fbc2fe72f72321bee1a87d66919b881a0336525a.tar.gz WARNING: Non existent bind point (directory) in container: '/project' WARNING: Non existent bind point (directory) in container: '/lsf' WARNING: Non existent bind point (directory) in container: '/share' WARNING: Non existent bind point (directory) in container: '/nl' Singularity: Invoking an interactive shell within container...

Singularity slamdunk:~> slamdunk map -r miRNA.fa -o ./ -5 5 -n 1 S13_trim.fastq Traceback (most recent call last): File "/usr/local/bin/slamdunk", line 5, in from pkg_resources import load_entry_point File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2927, in @_call_aside File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2913, in _call_aside f(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2940, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 637, in _build_master return cls._build_from_requirements(requires) File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 650, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 834, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (numpy 1.8.1 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('numpy>=1.9.0'), set(['pandas']))

t-neumann commented 6 years ago

@xbing88

Looks like there's a problem in the current build. Try the official release for now

tobneu/slamdunk:v0.2.4

But I will have a look at this soonish.

t-neumann commented 6 years ago

Build fixed on docker hub

aparajitadutta commented 4 years ago

Where exactly should we keep the fasta, bed and fastq files. I keep getting the following error when running it from py2_env virtual enviromnent:

Traceback (most recent call last): File "/home/aparajita/py2_env/bin/slamdunk", line 8, in sys.exit(run()) File "/home/aparajita/py2_env/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 478, in run runAll(args) File "/home/aparajita/py2_env/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 233, in runAll runMap(tid, bam, referenceFile, n, args.trim5, args.maxPolyA, args.quantseq, args.endtoend, args.topn, sampleInfo, dunkPath, args.skipSAM) File "/home/aparajita/py2_env/lib/python2.7/site-packages/slamdunk/slamdunk.py", line 148, in runMap mapper.Map(inputBAM, referenceFile, outputSAM, getLogFile(outputLOG), quantseqMapping, endtoendMapping, threads=threads, trim5p=trim5p, maxPolyA=maxPolyA, topn=topn, sampleId=tid, sampleName=sampleName, sampleType=sampleType, sampleTime=sampleTime, printOnly=printOnly, verbose=verbose) File "/home/aparajita/py2_env/lib/python2.7/site-packages/slamdunk/dunks/mapper.py", line 101, in Map if(checkStep([inputReference, inputBAM], [replaceExtension(outputSAM, ".bam")], force)): File "/home/aparajita/py2_env/lib/python2.7/site-packages/slamdunk/utils/misc.py", line 156, in checkStep raise RuntimeError("One or more input files don't exist!!!!: " + str(inFiles)) RuntimeError: One or more input files don't exist!!!!: ['GRCh38.p10.genome.fa', '1-G7-DMSO-1_S22_L002_R1_001.fastq.gz']

t-neumann commented 4 years ago

Since you created a new issue, I would let this one closed.