bwa-meth index error

[almanza-d]

I kept getting an error when running the indexing through the bwa-meth pipelien:

toolshed.files.ProcessException: bwa index -a bwtsw

When running independently the underlying command in bwa-meth, i.e. bwa index -a bwtsw *fa.bwameth.c2t, indexing works fine.

Any insight into this error?

[brentp]

Hi, can you show the full command and the full output?

[brentp]

It could be that you don't have write permission in the directory where the fasta files are located.

[almanza-d]

Here is the output:

converting c2t in hg19_lambda_puc19.fa to hg19_lambda_puc19.fa.bwameth.c2t
indexing with bwa-mem: hg19_lambda_puc19.fa.bwameth.c2t
[bwa_index] Pack FASTA... 43.38 sec
[bwa_index] Reverse the packed sequence... 17.29 sec
[bwa_index] Construct BWT for the packed sequence...
TextLengthFromBytePacked(): text length > 2^32!
cmd was:bwa index -a bwtsw hg19_lambda_puc19.fa.bwameth.c2t
return code was:1
Traceback (most recent call last):
  File "/drive3/dalmanza/software/bwa-meth/bwameth.py", line 559, in <module>
    main(sys.argv[1:])
  File "/drive3/dalmanza/software/bwa-meth/bwameth.py", line 510, in main
    sys.exit(bwa_index(convert_fasta(args[1])))
  File "/drive3/dalmanza/software/bwa-meth/bwameth.py", line 225, in bwa_index
    run("bwa index -a bwtsw %s" % fa)
  File "/drive3/dalmanza/software/bwa-meth/bwameth.py", line 97, in run
    list(nopen("|%s" % cmd.lstrip("|")))
  File "/home/diehnlab/dalmanza/.conda/envs/Diego/lib/python3.7/site-packages/toolshed/files.py", line 61, in process_iter
    raise ProcessException(cmd)
toolshed.files.ProcessException: bwa index -a bwtsw hg19_lambda_puc19.fa.bwameth.c2t

[brentp]

does hg19_lambda_puc19.fa contain more than 4GB of sequence? It seems so.

[almanza-d]

Does not seem like it.

3.0G    hg19_lambda_puc19.fa

[brentp]

and how big is the c2t file?

[almanza-d]

6.0G    hg19_lambda_puc19.fa.bwameth.c2t

[brentp]

can you show the content of hg19_lambda_puc19.fa.fai ?

[brentp]

and make sure you have the newest version of bwa installed?

[brentp]

and can you show the full command that you are running and the full output.

[almanza-d]

bwa version:

Version: 0.5.9-r16

Command ran:

bwameth.py index $REF

per your guide here. Full output is what I sent above. Are you looking for something different?

Output of hg19_lambda_puc19.fa.fai:

chrM    16571   6       50      51
chr1    249250621       16915   50      51
chr2    243199373       254252555       50      51
chr3    198022430       502315922       50      51
chr4    191154276       704298807       50      51
chr5    180915260       899276175       50      51
chr6    171115067       1083809747      50      51
chr7    159138663       1258347122      50      51
chr8    146364022       1420668565      50      51
chr9    141213431       1569959874      50      51
chr10   135534747       1713997581      50      51
chr11   135006516       1852243030      50      51
chr12   133851895       1989949684      50      51
chr13   115169878       2126478624      50      51
chr14   107349540       2243951907      50      51
chr15   102531392       2353448445      50      51
chr16   90354753        2458030472      50      51
chr17   81195210        2550192328      50      51
chr18   78077248        2633011450      50      51
chr19   59128983        2712650250      50      51
chr20   63025520        2772961820      50      51
chr21   48129895        2837247858      50      51
chr22   51304566        2886340358      50      51
chrX    155270560       2938671022      50      51
chrY    59373566        3097047000      50      51
chr1_gl000191_random    106433  3157608060      50      51
chr1_gl000192_random    547496  3157716644      50      51
chr4_gl000193_random    189789  3158275112      50      51
chr4_gl000194_random    191469  3158468719      50      51
chr7_gl000195_random    182896  3158664040      50      51
chr8_gl000196_random    38914   3158850616      50      51
chr8_gl000197_random    37175   3158890331      50      51
chr9_gl000198_random    90085   3158928272      50      51
chr9_gl000199_random    169874  3159020181      50      51
chr9_gl000200_random    187035  3159193475      50      51
chr9_gl000201_random    36148   3159384273      50      51
chr11_gl000202_random   40103   3159421167      50      51
chr17_gl000203_random   37498   3159462096      50      51
chr17_gl000204_random   81310   3159500367      50      51
chr17_gl000205_random   174588  3159583327      50      51
chr17_gl000206_random   41001   3159761430      50      51
chr18_gl000207_random   4262    3159803275      50      51
chr19_gl000208_random   92689   3159807646      50      51
chr19_gl000209_random   159169  3159902212      50      51
chr21_gl000210_random   27682   3160064588      50      51
chr4_ctg9_hap1  590426  3160092840      50      51
chr6_apd_hap1   4622290 3160695090      50      51
chr6_cox_hap2   4795371 3165409841      50      51
chr6_dbb_hap3   4610396 3170301135      50      51
chr6_mann_hap4  4683263 3175003755      50      51
chr6_mcf_hap5   4833398 3179780699      50      51
chr6_qbl_hap6   4611984 3184710780      50      51
chr6_ssto_hap7  4928567 3189415020      50      51
chr17_ctg5_hap1 1680828 3194442176      50      51
chrUn_gl000211  166566  3196156637      50      51
chrUn_gl000212  186858  3196326551      50      51
chrUn_gl000213  164239  3196517163      50      51
chrUn_gl000214  137718  3196684703      50      51
chrUn_gl000215  172545  3196825192      50      51
chrUn_gl000216  172294  3197001204      50      51
chrUn_gl000217  172149  3197176960      50      51
chrUn_gl000218  161147  3197352568      50      51
chrUn_gl000219  179198  3197516954      50      51
chrUn_gl000220  161802  3197699752      50      51
chrUn_gl000221  155397  3197864807      50      51
chrUn_gl000222  186861  3198023328      50      51
chrUn_gl000223  180455  3198213943      50      51
chrUn_gl000224  179693  3198398024      50      51
chrUn_gl000225  211173  3198581327      50      51
chrUn_gl000226  15008   3198796740      50      51
chrUn_gl000227  128374  3198812065      50      51
chrUn_gl000228  129120  3198943023      50      51
chrUn_gl000229  19913   3199074742      50      51
chrUn_gl000230  43691   3199095070      50      51
chrUn_gl000231  27386   3199139651      50      51
chrUn_gl000232  40652   3199167601      50      51
chrUn_gl000233  45941   3199209083      50      51
chrUn_gl000234  40531   3199255959      50      51
chrUn_gl000235  34474   3199297317      50      51
chrUn_gl000236  41934   3199332497      50      51
chrUn_gl000237  45867   3199375286      50      51
chrUn_gl000238  39939   3199422087      50      51
chrUn_gl000239  33824   3199462841      50      51
chrUn_gl000240  41933   3199497358      50      51
chrUn_gl000241  42152   3199540146      50      51
chrUn_gl000242  43523   3199583158      50      51
chrUn_gl000243  43341   3199627568      50      51
chrUn_gl000244  39929   3199671792      50      51
chrUn_gl000245  36651   3199712536      50      51
chrUn_gl000246  38154   3199749937      50      51
chrUn_gl000247  36422   3199788871      50      51
chrUn_gl000248  39786   3199826038      50      51
chrUn_gl000249  38502   3199866636      50      51
NC_001416.1     48502   3199905967      70      71
pUC19   2686    3199955267      80      81

[brentp]

ok. it looks like you just need to update your version of bwa. you can see latest release here: https://github.com/lh3/bwa/releases/tag/v0.7.17