search

kaneplusplus / bigmemory

126 stars 24 forks source link

Can we replace bigmemory with a small ALTREP? #74

Open kaneplusplus opened 7 years ago

cdeterman commented 7 years ago

Not sure, certainly an interesting question. The 'example' package, simplemmap, demonstrating this can be found here.

privefl commented 7 years ago

I found this presentation.

Maybe @ltierney could give some insight.

kaneplusplus commented 7 years ago

@cdeterman and @privefl after looking at the slides I think we can do this and I like the idea of making a big.matrix a first-class R object. I guess the question now is should we look closer at doing this? If we should, what will this look like? Will it be better than the current package or are we essentially reimplementing what we currently have?

cdeterman commented 7 years ago

I don't think it is a re-implementation as the 'shared memory' aspect of bigmemory is rather unique. I think this ALTREP representation is just finding a way to make the objects more compact.

kaneplusplus commented 7 years ago

@cdeterman It has been marketed for compact object representation but overriding duplicate could be used so that copied big.matrix objects can have copy-on-write behavior. If you are not going to make too many changes to a mmaped object, you could even cause a copied object to trigger a private mapping.

The vignette is also a good resource.