wheaton5
commented
4 years ago I probably know the issue, but not 100%.
Singularity only mounts the directories downstream of the working directory from which you run it. So if if you are running from /this/path and you are pointing at files in /this/otherpath/... or /otherpath/... it will not see that file. It will only see files in /this/path/. or /this/path/whatever/...
Also I don't know your cluster situation, but most scientific computing clusters will have head nodes which you are login to and then worker nodes on which large jobs are to be run. In order to run a large job such as souporcell you need to submit it to a cluster job scheduler such as SGE or LSF or SLURM. You may want to ask your system administrators/help desk about your particular system. Depending on your cluster settings you might be able to get away with running it on a head node but to reduce memory usage (usually limited on head nodes) and threads (also commonly limited on head nodes) you could run with --skip_remap True --common_variants <1kgenomes common variants linked in my readme> and --num_threads 4
As to your question of how I choose num threads, I normally choose 8, more will be faster but as not every stage is able to use all threads available, there will be some amount of diminishing returns with higher threads. Also in a job scheduler system you are gonna get an 8 thread job pretty fast but might have to wait a long time to even start a 32 thread job.
I hope this is helpful and I'm happy to answer any more question.
Best, Haynes
brekk129
hello wheaton5 -
I am trying to use your program to cluster a single cell RNA seq sample that should have cells from two genotypes. I am running it on my institute's supercomputing machine (remotely) using terminal on my mac. I'm getting this error - do you know what could be going on? do I need to run this within python in my terminal?
I used this form after installing singularity - singularity exec /path/to/souporcell.sif souporcell_pipeline.py -i /path/to/possorted_genome_bam.bam -b /path/to/barcodes.tsv -f /path/to/reference.fasta -t num_threads_to_use -o output_dir_name -k num_clusters
I used num_threads = 8 (how do you choose this?) num_clusters = 2 (should be two "genotypes")
checking modules imports done checking bam for expected tags Traceback (most recent call last): File "/opt/souporcell/souporcell_pipeline.py", line 59, in
for (index, line) in enumerate(barcodes):
File "/usr/local/envs/py36/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)
thanks I would appreciate any help! I'm an immunologist trying to run my own code. could also email me - jriedl@umn.edu