Index out of bounds

[samlipworth]

Trying to cluster an admittedly enourmous alignment (~2700 isolates, snps only) - I have succesfully done a slightly smaller version of this recently so I'm not sure what's wrong but I'm getting the following error - any ideas?

`

s<-read.FASTA('./out') library(fastbaps) s<-import_fasta_sparse_nt(s) baps.hc<-fast_baps(s) [1] "Calculating initial clustering..." [1] "Calculating initial dk values..." [1] "Clustering using hierarchical Bayesian clustering..." Error in bhier_parallel(sparse.data, initial.partition, dk.initial, n.cores) : Mat::operator(): index out of bounds `

[gtonkinhill]

Hi,

Sorry for being slow to get to this. Were you able to resolve the issue? If not how many columns were in the alignment? I'm wondering if this is a memory issue and if so knowing how large the alignment is should let me reproduce the problem.

[YiweiZhu]

Hi, @gtonkinhill I got the same issue. My MSA file includes 374 sequences and each sequence is 559676-bp-long (SNPs only).

[gtonkinhill]

Hi,

Sorry for the delay, I was not able to reproduce the issue with a matrix of that size. However, I did run into an issue with the function that loads the MSA directly from a file. Does loading using ape resolve the problem?

msa <- ape::read.FASTA("my_msa.fasta")
sp_data <- fastbaps::import_fasta_sparse_nt(msa)

I will look into what causes the issue with importing directly from an MSA.

If this does not fix the issue would you be able to send me an example of how you are running fastbaps? For reference I've copied the code I am using to attempt to reproduce the issue. It generates a random MSA of the same size so if you are able to reproduce the problem using this simulated data that would help a lot in finding the source of the bug.

library(fastbaps)
library(ape)

#Generate random data
aln_len <- 559676
nseqs <- 374
msa <- matrix(rep(sample(c('A','C','G','T'), replace = TRUE, size = aln_len), nseqs),
                             nrow = nseqs, byrow = TRUE)
nvar <- ceiling(0.02*aln_len*nseqs)
msa[sample(1:length(msa), size = nvar)] <- sample(c('A','C','G','T'), replace = TRUE, size = nvar)
msa <- ape::as.DNAbin(msa)
ape::write.FASTA(msa, "temp.fasta")

# Run fastbaps
msa <- ape::read.FASTA("temp.fasta")
sp_data <- fastbaps::import_fasta_sparse_nt(msa)
fb <- fastbaps::fast_baps(sp_data)

[jananiharan]

Same issue. I have 32 sequences that are 497993 bp long. I got SNPs from the genome alignment using snp-sites and ran the following commands:

msa <- ape::read.FASTA('wa_ak_aligned_snps.fasta')
sparse.data <- fastbaps::import_fasta_sparse_nt(msa)

fb <- fastbaps::fast_baps(sparse.data)
[1] "Calculating initial clustering..."
[1] "Calculating initial dk values..."
[1] "Clustering using hierarchical Bayesian clustering..."
Error in bhier_parallel(sparse.data, initial.partition, dk.initial, n.cores) : 
  Mat::operator(): index out of bounds

sparse.data.opt <- optimise_prior(sparse.data, type = "optimise.symmetric")
Error in tree_llk(temp.sparse.data, temp.h$merge) : 
  Mat::operator(): index out of bounds

[gtonkinhill]

Hi,

Thanks confirming this is still an issue. I have been having trouble reproducing this issue. Would it be possible for you to send me a dataset that reproduces the problem?

My email is gt4@sanger.ac.uk

[gtonkinhill]

Hi Janani,

Thanks very much for sending through the alignment. It turns out this was an issue with the matrix loading function which relied on ape.

Thus, fastbaps::import_fasta_sparse_nt(msa) would fail while fastbaps::import_fasta_sparse_nt('wa_ak_aligned_snps.fasta') would have worked.

However, the later had another issue which caused a stack overflow error with large alignments.

Both of these issues should now be fixed in version 1.0.6

[jananiharan]

Can confirm that v 1.0.6 solves this issue. Thanks!