search

hannes / MonetDBLite

MonetDB reconfigured as a library
108 stars 11 forks source link

MonetDBLite store database in memory #117

Closed ssssman closed 7 years ago

ssssman commented 7 years ago

My understanding is that MonetDBLite save the database in the local directory. Is it possible to create database in memory just like sqlite?

https://www.sqlite.org/inmemorydb.html

ajdamico commented 7 years ago

good question.. db <- dbConnect( MonetDBLite::MonetDBLite() ) works the same way in terms of the R syntax but still stores that temporary database on the disk inside of tempdir(). is there a reason it's important not to touch the local disk at all? thanks

hannes commented 7 years ago

Yes, this is possible with a ramdisk. MonetDB relies on file system operations (mv) for its atomic transaction commits. We could automate this in the future, it seems doable at least on Linux.

ssssman commented 7 years ago

Running in memory could make monetDBlite/montetDB meet the speed requirement for real-time data analytics, like some other databases have provided, such as KDB+/q or Microsoft SQL database(Memory-optimized tables). Tables on hard-disk for historical data and Tables in memory for real-time data. Users don't need to deal with two different databases. At the end of day, the data in memory is just saved on the disk and become part of historical database. This could complete MonetDB/MonetDBLite and let people to build apps on them. Just my two cents.

hannes commented 7 years ago

This effectively already happens when the database table fit in RAM and are not updated frequently as the operating system caches frequently accessed disk pages in RAM.