kaneplusplus / bigmemory

126 stars 24 forks source link

"Put this matrix on disk" #116

Closed multimeric closed 5 months ago

multimeric commented 5 months ago

I'm trying to find the right function that will take an existing matrix m and convert it to a disk-backed one.

I've tried this, but it seems to fill the matrix with m[[1]] instead of copying all the data.

bigmemory::filebacked.big.matrix(
  nrow = nrow(m),
  ncol = ncol(m),
  init = m,
  ...
)

I've also tried this, and while it does return a big.memory object, it's not filebacked:

proxy <- bigmemory::as.big.matrix(m)
bigmemory::is.filebacked(proxy)
# [1] FALSE

The only working solution I can find is a bit ugly:

proxy <- bigmemory::filebacked.big.matrix(nrow = nrow(m), ncol=ncol(m), ...)
proxy[] <- m
bigmemory::is.filebacked(proxy)
# [1] TRUE
privefl commented 5 months ago

You probably need to specify backingfile and backingpath in as.big.matrix().

multimeric commented 5 months ago

Good point. I thought I had tried this but apparently not!

proxy <- bigmemory::as.big.matrix(m, backingpath = tempdir(), backingfile = "m.bin", descriptorfile = "m.desc")
bigmemory::is.filebacked(proxy)
# [1] TRUE