search

ijuric / MAPS

18 stars 11 forks source link

Correct syntax to define BWA index path #10

Closed daramirez closed 5 years ago

daramirez commented 5 years ago

Hello,

I am trying using MAPS, but I have problems running the test bin/run_pipeline_test.sh I hope that you can help me.

I have cloned the MAPS repository to this location: /Users/dara6367/repos/MAPS

I suspect that I am just not providing to the 'run_pipeline_test.sh' script the correct path to my BWA index files.

In 'run_pipeline_test.sh' I changed the following lines as shown:

python_path=/opt/python/3.6.3/bin/python3.6 Rscript_path=/opt/R/3.5.1/bin/Rscript fastq_dir="/Users/dara6367/repos/MAPS/examples/test_set1" outdir="/Users/dara6367/repos/MAPS/examples/test_set1/output" macs2_filepath="/Users/dara6367/repos/MAPS/examples/test_set1/macs2_peaks_final.replicated.narrowPeak" bwa_index="/scratch/Shares/chuong/genomes/mm10/bwa"

(I have tried also bwa_index="/scratch/Shares/chuong/genomes/mm10/bwa/" or bwa_index="/scratch/Shares/chuong/genomes/mm10/bwa/mm10" but they also do not work).

In the BWA mm10 index directory, I have the five BWA index files: mm10.amb mm10.ann mm10.bwt mm10.pac mm10.sa

After running the 'run_pipeline_test.sh' script, I see the error that I am attaching here. MAPS_error.txt

And digging in the few files that are created in the output folder, the feather bwa logs show this:

/Users/dara6367/repos/MAPS/examples/test_set1/output/feather_output/test_set1_20190702_155131/tempfiles/test_set1_R1.fastq.bwa.sam.log [E::bwa_idx_load_from_disk] fail to locate the index files

/Users/dara6367/repos/MAPS/examples/test_set1/output/feather_output/test_set1_20190702_155131/tempfiles/test_set1_R2.fastq.bwa.sam.log [E::bwa_idx_load_from_disk] fail to locate the index files

I also have BWA already in my path: echo $PATH /opt/bwa/0.7.15 /opt/samtools/1.8/bin /opt/bedtools/2.25.0/bin /opt/python/3.6.3/bin /opt/R/3.5.1/bin etc.

In addition, I have successfully tested that my BWA mm10 index files work by using BWA to align the two test fastq files myself directly, as shown here below. So I know that the index files are compatible with BWA.

bwa aln /scratch/Shares/chuong/genomes/mm10/bwa/mm10 /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R1.fastq > /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R1.sai

bwa aln /scratch/Shares/chuong/genomes/mm10/bwa/mm10 /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R2.fastq > /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R2.sai

bwa sampe /scratch/Shares/chuong/genomes/mm10/bwa/mm10 /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R1.sai /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R2.sai /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R1.fastq /Users/dara6367/repos/MAPS/examples/test_set1/test_set1_R2.fastq > /Users/dara6367/repos/MAPS/examples/test_set1/test_set1.sam

Could you please tell me how to properly set the path to my BWA index files? Thank you!

armenabnousi commented 5 years ago

Hi Daniel,

Looking at your scripts looks like the BWA index you are using for bwa aln is slightly different from the one you have in run_pipeline file. I think if you add the mm10 at the end of the bwa_index (bwa_index=/scratch/Shares/chuong/genomes/mm10/bwa/mm10) it will work. Hope this helps, please do let us know if it is still not working.

Thanks, Armen

On Tue, Jul 2, 2019 at 6:08 PM Daniel Ramirez notifications@github.com wrote:

Hello,

I am trying using MAPS, but I have problems running the test bin/run_pipeline_test.sh I hope that you can help me.

I have cloned the MAPS repository to this location: /Users/dara6367/repos/MAPS

I suspect that I am just not providing to the 'run_pipeline_test.sh' script the correct path to my BWA index files.

In 'run_pipeline_test.sh' I changed the following lines as shown:

python_path=/opt/python/3.6.3/bin/python3.6 Rscript_path=/opt/R/3.5.1/bin/Rscript fastq_dir="/Users/dara6367/repos/MAPS/examples/test_set1" outdir="/Users/dara6367/repos/MAPS/examples/test_set1/output"

macs2_filepath="/Users/dara6367/repos/MAPS/examples/test_set1/macs2_peaks_final.replicated.narrowPeak" bwa_index="/scratch/Shares/chuong/genomes/mm10/bwa"

After running the 'run_pipeline_test.sh' script, I see the error that I am attaching here. MAPS_error.txt https://github.com/ijuric/MAPS/files/3352481/MAPS_error.txt

And digging in the few files that are created in the output folder, the feather bwa logs show this:

/Users/dara6367/repos/MAPS/examples/test_set1/output/feather_output/test_set1_20190702_155131/tempfiles/test_set1_R1.fastq.bwa.sam.log [E::bwa_idx_load_from_disk] fail to locate the index files

/Users/dara6367/repos/MAPS/examples/test_set1/output/feather_output/test_set1_20190702_155131/tempfiles/test_set1_R2.fastq.bwa.sam.log [E::bwa_idx_load_from_disk] fail to locate the index files

In addition, I have successfully tested that my BWA mm10 index files work by using BWA to align the two test fastq files myself directly, as shown here below. So I know that the index files are compatible with BWA.

bwa aln /scratch/Shares/chuong/genomes/mm10/bwa/mm10 ~/repos/MAPS/examples/test_set1/test_set1_R1.fastq \

~/repos/MAPS/examples/test_set1/test_set1_R1.sai

bwa aln /scratch/Shares/chuong/genomes/mm10/bwa/mm10 ~/repos/MAPS/examples/test_set1/test_set1_R2.fastq \

~/repos/MAPS/examples/test_set1/test_set1_R2.sai

bwa sampe /scratch/Shares/chuong/genomes/mm10/bwa/mm10 ~/repos/MAPS/examples/test_set1/test_set1_R1.sai ~/repos/MAPS/examples/test_set1/test_set1_R2.sai ~/repos/MAPS/examples/test_set1/test_set1_R1.fastq ~/repos/MAPS/examples/test_set1/test_set1_R2.fastq \

/Users/dara6367/repos/MAPS/examples/test_set1/test_set1.sam

Could you please tell me how to properly set the path to my BWA index files? Thank you!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ijuric/MAPS/issues/10?email_source=notifications&email_token=AEFQ6XAUDWZLEMZBZ7FHX2DP5PGXDA5CNFSM4H5AGOYKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G47ZTWQ, or mute the thread https://github.com/notifications/unsubscribe-auth/AEFQ6XFLHKYOO4I3CDBWCDTP5PGXDANCNFSM4H5AGOYA .

daramirez commented 5 years ago

Hi Armen,

Thank you for your prompt reply!

I have tried using

bwa_index=/scratch/Shares/chuong/genomes/mm10/bwa/mm10

just as I have used those very index files with the command 'bwa aln', but it still does not work. The error is slightly different, as I am attaching here. MAPS_error2.txt

At the very top it says:

ERROR: Input file /scratch/Shares/chuong/genomes/mm10/bwa/mm10 does not exist. Please make sure the correct path is provided. Exiting!

Which is what confuses me because those files do exist, as I use the same line with 'bwa aln'.

Daniel

armenabnousi commented 5 years ago

Daniel, I do think that you have potentially found a bug in our code. To confirm, you have a files called /scratch/Shares/chuong/genomes/mm10/bwa/mm10.fai and /scratch/Shares/chuong/genomes/mm10/bwa/mm10.bwt, right? If you do have these files, do you also have a file called "/scratch/Shares/chuong/genomes/mm10/bwa/mm10"? if not, could you please generate an empty file with this name and try running the program. This should fix the problem and I'll change the code tomorrow to take care of this issue. Sorry about the complication.

Thanks, Armen

On Tue, Jul 2, 2019 at 8:52 PM Daniel Ramirez notifications@github.com wrote:

Hi Armen,

Thank you for your prompt reply!

I have tried using

bwa_index=/scratch/Shares/chuong/genomes/mm10/bwa/mm10

just as I have used those very index files with the command 'bwa aln', but it still does not work. The error is slightly different, as I am attaching here. MAPS_error2.txt https://github.com/ijuric/MAPS/files/3352806/MAPS_error2.txt

At the very top it says:

ERROR: Input file /scratch/Shares/chuong/genomes/mm10/bwa/mm10 does not exist. Please make sure the correct path is provided. Exiting!

Which is what confuses me because those files do exist, as I use the same line with 'bwa aln'.

Daniel

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ijuric/MAPS/issues/10?email_source=notifications&email_token=AEFQ6XHANR4JKTD26QDZXSDP5PZ25A5CNFSM4H5AGOYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZC6CKA#issuecomment-507896104, or mute the thread https://github.com/notifications/unsubscribe-auth/AEFQ6XF72H6OEQGG3ZKDL43P5PZ25ANCNFSM4H5AGOYA .

daramirez commented 5 years ago

Hi Armen,

Thank you very much with your help with this issue. You were right.

I created an empty file named 'mm10' in the same directory where the other 5 bwa index files live and I now can run 'run_pipeline_test.sh' successfully.

The output file /Users/dara6367/repos/MAPS/examples/test_set1/output/MAPS_output/test_set1_20190703_082143/test_set1.5k.2.peaks.bedpe

Is the same as /Users/dara6367/repos/MAPS/examples/expected_output_test_set1.bedpe

Also, I did not have a file 'mm10.fai' along my other index files, as you mentioned. I only had the following files below, and they are enough to run MAPS on that test config file.

mm10 0 mm10.amb 11K mm10.ann 2.9K mm10.bwt 2.6G mm10.pac 652M mm10.sa 1.3G

Thank you! Daniel

armensly commented 5 years ago

Glad it worked Daniel. We will modify the code to deal with this situation and push the new version this week. I'm closing this issue. Feel free to open a new issue if need be. Thanks! Armen