Jiefei-Wang / SharedObject

Sharing R objects across multiple R processes without duplicating the object in memory
42 stars 2 forks source link

Allocating an empty shared object #7

Closed Jiefei-Wang closed 3 years ago

Jiefei-Wang commented 4 years ago

This is a feature asked by Ashis Saha @alorchhota from the email

Hi Jiefei,

I have been using your SharedObject R package. The package is cool. 

I have a couple of questions about the package. I like to share a big matrix across multiple processes. So, I create a shared object in the following way.

mat <- matrix(0, nrow = 3, ncol = 3)        # pretend mat is a big matrix
shared_mat <- share(mat, copyOnWrite = F)   # shared matrix

Now, I have two copies of the big matrix. I like to remove mat to save memory.

rm(mat)
gc(reset = T)

Here are my questions:
1) Could removing mat somehow affect the shared_mat matrix? Specifically, by any chance, could shared_mat point to an invalid memory after removing mat?
2) Is there a way to create a big shared matrix without creating two copies and then deleting one? If so, I don't have to keep two big matrices in memory at the same time.

For your convenience, I attached the code of a toy example of adding first 10 integers using SharedObject. Any help would be highly appreciated. Thanks!

Best,
Ashis
Jiefei-Wang commented 3 years ago

Hi @alorchhota, the devel version in Bioconductor now supports creating an empty shared object. Please see ?SharedObject for the details.

Cheers, Jiefei