OSError

[caleblareau]

Hi, I tried running BRAPeS for ~ 300 cells and execution stopped on my first cell with the following error message... any idea what to do here? maybe an RSEM issue?

2019-05-27 14:30:01.052223 Reconstructing heavy chains
2019-05-27 14:30:28.222468 Reconstructing kappa chains
2019-05-27 14:30:28.236398 Reconstructing lambda chains
2019-05-27 14:30:29.412734 Creating full BCR sequences
2019-05-27 14:30:29.583546 Running RSEM to quantify expression of all possible isotypes
2019-05-27 14:30:29.587537 Did not reconstruct any heavy chains, not running RSEM on this chain
2019-05-27 14:30:29.587571 Did not reconstruct any kappa chains, not running RSEM on this chain
Traceback (most recent call last):
  File "BRAPeS/brapes.py", line 2227, in <module>
    args.HVR_score, args.HVR_min_reads, args.HVR_max_reads, args.HVR_moveOn)
  File "BRAPeS/brapes.py", line 59, in runBCRpipe
    HVR_moveOn, hvr_script, hvr_path, genome, ighv, igkv, iglv)
  File "BRAPeS/brapes.py", line 347, in runSingleCell
    runRsem(outDir, rsem, bowtie2, fullTcrFileHeavy, fullTcrFileKappa,fullTcrFileLambda, output, samtools)
  File "BRAPeS/brapes.py", line 1279, in runRsem
    runRSEMonOneFile(fullTcrFileLambda, rsem, bowtie2, rsemIndDir, output, samtools, "lambda")
  File "BRAPeS/brapes.py", line 1297, in runRSEMonOneFile
    '-q', resF, rsemIndDir + '/VDJ.' + chain + '.seq'], stdout=devnull, stderr=devnull)
  File "/apps/source/python-2.7.3/Python-2.7.3/lib/python2.7/subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/apps/source/python-2.7.3/Python-2.7.3/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/apps/source/python-2.7.3/Python-2.7.3/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

[shakea02]

Hi Caleb, Is it possible that RSEM is not in your default PATH (or you provided the wrong path with the -rsem parameter)? That could be the issue. If RSEM is indeed in your PATH, could you send me the commend line you used to run BRAPeS? That might help me understand what the problem is.

Best, Shaked

[caleblareau]

I aligned with STAR, so I supplied the same bam files twice. This was my command.

python2 BRAPeS/brapes.py -path aligned -g hg38 -o brapes_ss2 -sumF brapes_ss2 -bam aligned.bam -unmapped aligned.bam

I do have rsem in my PATH

[shakea02]

Do you also have bowtie2 in the PATH? If you don't that could cause bowtie2 to fail. Also, what is under your "aligned" directory? Does it have a subfolder for each cell, or is the folder only with data of a single cell? Are you running this command once or in parallel for several cells?

[caleblareau]

bowtie2 is in the PATH. The file structure looks like this:

aligned/GMLCL_108_S108:
-rw-rw----. 1 cl322 aryee 96650028 May 27 12:38 aligned.bam
-rw-rw----. 1 cl322 aryee  1873136 May 27 12:38 aligned.bam.bai
-rw-rw----. 1 cl322 aryee     1848 May 27 12:38 GMLCL_108_S108Log.final.out
-rw-rw----. 1 cl322 aryee 16491017 May 27 12:38 GMLCL_108_S108Log.out
-rw-rw----. 1 cl322 aryee      447 May 27 12:38 GMLCL_108_S108Log.progress.out
-rw-rw----. 1 cl322 aryee      434 May 27 12:38 GMLCL_108_S108Log.std.out
-rw-rw----. 1 cl322 aryee   374973 May 27 12:38 GMLCL_108_S108ReadsPerGene.out.tab
-rw-rw----. 1 cl322 aryee  1311551 May 27 12:38 GMLCL_108_S108SJ.out.tab

aligned/GMLCL_109_S109:
-rw-rw----. 1 cl322 aryee   175026 May 27 12:38 aligned.bam
-rw-rw----. 1 cl322 aryee  1449704 May 27 12:38 aligned.bam.bai
-rw-rw----. 1 cl322 aryee     1822 May 27 12:38 GMLCL_109_S109Log.final.out
-rw-rw----. 1 cl322 aryee 16490899 May 27 12:38 GMLCL_109_S109Log.out
-rw-rw----. 1 cl322 aryee      447 May 27 12:38 GMLCL_109_S109Log.progress.out
-rw-rw----. 1 cl322 aryee      434 May 27 12:38 GMLCL_109_S109Log.std.out
-rw-rw----. 1 cl322 aryee   353334 May 27 12:38 GMLCL_109_S109ReadsPerGene.out.tab
-rw-rw----. 1 cl322 aryee     1969 May 27 12:38 GMLCL_109_S109SJ.out.tab

** for about 300 cells**

So the idea was a subfolder for each cell. I had attempted to run the command once rather than multiple times.

[shakea02]

Ok, good. You're supposed to only run it once, this error could happen if running BRAPeS multiple times in parallel while writing to the same output folder.

In the cell in which BRAPeS crashed on, is there a file with fasta sequences named brapes_ss2.lambda.full.BCRs.fa ? It's possible that this file (or lack of it) is what makes BRAPeS crash

[caleblareau]

I see. Thanks for the suggestion.

So BRAPeS crashed on the very first file. I don’t see any output, specifically not this fasta file that you are referencing.

On May 27, 2019, at 5:07 PM, shakea02 notifications@github.com wrote:

Ok, good. You're supposed to only run it once, this error could happen if running BRAPeS multiple times in parallel while writing to the same output folder.

In the cell in which BRAPeS crashed on, is there a file with fasta sequences named brapes_ss2.lambda.full.BCRs.fa ? It's possible that this file (or lack of it) is what makes BRAPeS crash

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/YosefLab/BRAPeS/issues/1?email_source=notifications&email_token=AD32FYMCIU6BL3CUIUOLPU3PXRES3A5CNFSM4HP52ET2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWKQNSY#issuecomment-496305867, or mute the thread https://github.com/notifications/unsubscribe-auth/AD32FYIBZ5BJ42IZXGDQMF3PXRES3ANCNFSM4HP52ETQ.

[shakea02]

Can you send me the full log of the run? If BRAPeS doesn't write any output I think there might be a problem with how BRAPeS interrupt the parameters

[caleblareau]

full log here a majority of it is warnings about read mate mappings, which I can figure out some other time

(thanks by the way for the active debugging help!)

[shakea02]

Of course, happy to help! The read mate mappings warnings means that some of the reads are mapped to both strands in different mapping, so BRAPeS can't infer the correct orientation of their unmapped pair. That could be an issue if that happens to most of the reads, but I don't think it's the problem here.

Do you see any new files created under the folder aligned/GMLCL_02_S2 ? Specifically files with the prefix brapes_ss2 ? If you don't see any it means that BRAPeS is writing the files to the incorrect directory. Could you try running it again but for the -path parameter give it the full directory instead of the relative directory?

[caleblareau]

Makes sense regarding the umapped pairs

I do in fact see new files...

-rw-rw----. 1 cl322 aryee      713 May 27 17:14 brapes_ss2.heavy.junctions.txt
-rw-rw----. 1 cl322 aryee  1274020 May 27 17:14 brapes_ss2.heavy.R1.fa
-rw-rw----. 1 cl322 aryee  1274020 May 27 17:14 brapes_ss2.heavy.R2.fa
-rw-rw----. 1 cl322 aryee   676383 May 27 17:14 brapes_ss2.heavy.mapped.and.unmapped.fa
-rw-rw----. 1 cl322 aryee        0 May 27 17:14 brapes_ss2.kappa.junctions.txt
-rw-rw----. 1 cl322 aryee        0 May 27 17:14 brapes_ss2.kappa.mapped.and.unmapped.fa
-rw-rw----. 1 cl322 aryee        0 May 27 17:14 brapes_ss2.kappa.R1.fa
-rw-rw----. 1 cl322 aryee        0 May 27 17:14 brapes_ss2.kappa.R2.fa
-rw-rw----. 1 cl322 aryee      308 May 27 17:14 brapes_ss2.lambda.junctions.txt
-rw-rw----. 1 cl322 aryee    23600 May 27 17:14 brapes_ss2.lambda.R1.fa
-rw-rw----. 1 cl322 aryee    23600 May 27 17:14 brapes_ss2.lambda.R2.fa
-rw-rw----. 1 cl322 aryee    30408 May 27 17:14 brapes_ss2.lambda.mapped.and.unmapped.fa
-rw-rw----. 1 cl322 aryee      743 May 27 17:15 brapes_ss2.reconstructed.junctions.heavy.fa
-rw-rw----. 1 cl322 aryee        0 May 27 17:15 brapes_ss2.reconstructed.junctions.kappa.fa
-rw-rw----. 1 cl322 aryee      210 May 27 17:15 brapes_ss2.reconstructed.junctions.lambda.fa
-rw-rw----. 1 cl322 aryee     9803 May 27 17:15 brapes_ss2.lambda.full.BCRs.fa
drwxrws---. 2 cl322 aryee     4096 May 27 17:15 rsem_ind453604

Re-running with the absolute directory didn't change anything.

[shakea02]

That's good. So for some reason there's a problem creating an RSEM index from the reconstructed sequences in the brapes_ss2.lambda.full.BCRs.fa file. Can you try running the following command from the commend line:

rsem-prepare-reference --bowtie2 full/Path/to/aligned/GMLCL_02_S2/brapes_ss2.lambda.full.BCRs.fa /full/Path/to/aligned/rsem_ind453604/VDJ.lambda.seq

and send me the log? It could tell if the problem is with RSEM itself or with how BRAPeS calls RSEM.

[caleblareau]

The version of RSEM that I'm using evidently doesn't have a --bowtie2 flag available based on this suggestion. Let me try updating and rerunning things. Thanks for the great suggestion.

[shakea02]

Sure thing! Let me know if how it goes