search

DiskFrame / disk.frame

Fast Disk-Based Parallelized Data Manipulation Framework for Larger-than-RAM Data
https://diskframe.com
Other
595 stars 40 forks source link

solve function doesn't work with this package for large observation of matrix #383

Closed rohitwalthare closed 2 years ago

rohitwalthare commented 2 years ago

I am trying to use the solve function with the matrix which consists of 49543x16 observation and trying to use the following command but it isn't working "diag(X %% solve(XX) %% t(X))" also the error is similar with memory allocation as "cannot allocate vector of size ..."

xiaodaigh commented 2 years ago

Disk.frame can’t be used to solve massive matrix problems like this unfortunately.

Only table problems.

rohitwalthare commented 2 years ago

is there any other possible solution for this to solve, cause I'm trying to use whatever the available solution we have over the web and tried basically, so thought to give the suggestion for further development?

really appreciate your quick reply to the earlier thread and thank you.

xiaodaigh commented 2 years ago

Not sure. But I imagine you would need some big memory package Check bigmemory? Or you might need some disk based mmapping approach. Yeah, I am probably not much help beyond that.

rohitwalthare commented 2 years ago

thank you for your suggestions, though package bigmemory didn't help.