search

statgen / bamUtil

http://genome.sph.umich.edu/wiki/BamUtil
87 stars 29 forks source link

Failed to open bam fille #15

Closed ghost closed 8 years ago

ghost commented 8 years ago

Hi, I am a newbie in Github and analyzing NGS data using command line, I hope I am writing this question in the right place. i am trying to run clipoverlap on some bam files. I get an error: FAIL_IO: Failed to Open "THE BAM FILE NAME". The directory if right and I can not figure out why I am getting this error.

pjvandehaar commented 8 years ago

Yep, this is the right place. Could you tell us a little more about what you're doing? What's the exact command you're running?

ghost commented 8 years ago

This is the exact command: genesis/home/amohajeri/src/bamUtil/bin/bam clipOverlap --in ../bam-files/MCL-126N1.bam --out ./MCL-126N1-clov.bam

I am using mqsub to run several of these command for different bam files.

pjvandehaar commented 8 years ago

Whether you respond through email or Github, the response goes to both places. Either works.

Is that the full error FAIL_IO: Failed to Open ../bam-files/MCL-126N1.bam? Is there more to work from? Sorry, I'm not the programmer for bamUtil, so I need a little more help. :)

ghost commented 8 years ago

I think that was the full error. My bash was acting strange tonight and I was getting the lines mixed together so there might be some more information that I couldn't see because of the mix of lines. I will check this again tomorrow afternoon and get back to you hopefully with more information. :)

----- Original Message ----- From: "Peter VandeHaar" notifications@github.com To: "statgen/bamUtil" bamUtil@noreply.github.com Cc: "amohajeri" arezoo_mohajeri@sfu.ca Sent: Monday, October 26, 2015 9:25:01 PM Subject: Re: [bamUtil] Failed to open bam fille (#15)

Whether you respond through email or Github, the response goes to both places. Either works.

Is that the full error FAIL_IO: Failed to Open ../bam-files-MCL-126N1.bam., or something like that? Is there more to work from? Sorry, I'm not the programmer for bamUtil, so I need a little more help. :)

Reply to this email directly or view it on GitHub: https://github.com/statgen/bamUtil/issues/15#issuecomment-151371524

Best regards, Arezoo Mohajeri

pjvandehaar commented 8 years ago

Can you run genesis/home/amohajeri/src/bamUtil/bin/bam validate --so_coord --in ../bam-files/MCL-126N1.bam to see whether bamUtil can read your file and it's in order?

If bamUtil's output has lines that are strangely mixed together more often, that would be good to know, because we could fix that if we knew when it happened.

ghost commented 8 years ago

I did what you suggested and bamUtil successfully read the file. Can the issue be with the mqsub or using batch file?

ghost commented 8 years ago

This is the output for using --so_coord:

TotalReads(e6)  105.03
MappedReads(e6) 99.92
PairedReads(e6) 105.03
ProperPair(e6)  98.85
DuplicateReads(e6)  43.58
QCFailureReads(e6)  9.57

MappingRate(%)  95.14
PairedReads(%)  100.00
ProperPair(%)   94.12
DupRate(%)  41.50
QCFailRate(%)   9.12

TotalBases(e6)  13128.24
BasesInMappedReads(e6)  12489.92
Returning: 0 (SUCCESS)
pjvandehaar commented 8 years ago

Could you run bam |& head -n3? If that fails, just bam.

pjvandehaar commented 8 years ago

Could you run genesis/home/amohajeri/src/bamUtil/bin/bam clipOverlap --in ../bam-files/MCL-126N1.bam --out ./MCL-126N1-clov.bam; echo $?

ghost commented 8 years ago

There were three problems in my batch file: First in the name of the file which is ../bam-files/MCL-126N.bam and second I used a short directory instead of full directory and third there was a dictation mistake of using 0 instead of O in word clipOverlap). I am trying the bamUtil again fixing the three mistake and I am waiting for the results. If I get the same error I will try the two commands you suggested me and write you back the results.

ghost commented 8 years ago

Hi Peter, Is there a way to test if bamUtil worked successfully? It seems that it worked and I didn't get the error I was getting before. Probably those mistakes were the issue.

pjvandehaar commented 8 years ago

Sorry, I don't know how to verify your output. Maybe check that your output file is between 50% and 99% the size of the input file?

Sounds to me like it worked fine.

ghost commented 8 years ago

Thanks. The size is the same between input and output. Perhaps there were not much reads removed.

pjvandehaar commented 8 years ago

Good to hear. I'm glad it's working for you now.

mktrost commented 8 years ago

Sorry for the delay in my response. The file size after clipOverlap should remain about the same as the original size. No reads are removed, just the CIGAR string will change to soft clip the overlapping bases in 1 of the reads in an overlapping pair.

On success, "Completed ClipOverlap Successfully." should have been written to stderr.

You can use bam diff to compare your two bam files: genesis/home/amohajeri/src/bamUtil/bin/bam diff--in1 ../bam-files/MCL-126N1.bam --in2 ./MCL-126N1-clov.bam --all |head

Use head to only see the first few differences. You should see differences in the CIGARs between the two bam files. If you don't, then you probably didn't have any overlaps.

Does that help?

And yes, as you figured out, when sending your command to a remote machine you probably need to pass full paths rather than relative paths.

Let me know if you have any additional questions/problems.

Mary Kate Wing

ghost commented 8 years ago

Hi Mary,

Thank you for the tips, it was very helpful and informative. I used both stderr and diff. Both confirmed that the ClipOverlap was successful.

Arezoo

----- Original Message ----- From: "mktrost" notifications@github.com To: "statgen/bamUtil" bamUtil@noreply.github.com Cc: "amohajeri" arezoo_mohajeri@sfu.ca Sent: Tuesday, October 27, 2015 5:11:14 PM Subject: Re: [bamUtil] Failed to open bam fille (#15)

Sorry for the delay in my response. The file size after clipOverlap should remain about the same as the original size. No reads are removed, just the CIGAR string will change to soft clip the overlapping bases in 1 of the reads in an overlapping pair.

On success, "Completed ClipOverlap Successfully." should have been written to stderr.

You can use bam diff to compare your two bam files :

genesis/home/amohajeri/src/bamUtil/bin/bam diff--in1 ../bam-files/MCL-126N1.bam --in2 ./MCL-126N1-clov.bam --all |head

Use head to only see the first few differences. You should see differences in the CIGARs between the two bam files. If you don't, then you probably didn't have any overlaps.

Does that help?

And yes, as you figured out, when sending your command to a remote machine you probably need to pass full paths rather than relative paths.

Let me know if you have any additional questions/problems.

Mary Kate Wing

On Oct 27, 2015 7:35 PM, "amohajeri" notifications@github.com wrote:

Thanks. The size is the same between input and output. Perhaps there were not much reads removed.

— Reply to this email directly or view it on GitHub https://github.com/statgen/bamUtil/issues/15#issuecomment-151675377.

Reply to this email directly or view it on GitHub: https://github.com/statgen/bamUtil/issues/15#issuecomment-151680769

Best regards, Arezoo Mohajeri Hi Mary,