search

composewell / streamly

High performance, concurrent functional programming abstractions
https://streamly.composewell.com
Other
865 stars 66 forks source link

Anonymous or temporary files #230

Open harendra-kumar opened 5 years ago

harendra-kumar commented 5 years ago

An anonymous file is just like an array and will have the identical APIs. Just like an array is an anonymous chunk of memory that is freed automatically, anonymous files are chunks of filesystem space that have no name (for the programmer) and are freed automatically. The only difference is that files start out on the filesystem and arrays start out in memory (they can get swapped to filesystem though on memory pressure). This can be useful when we want to temporarily store large amounts of data without putting pressure on memory.

This can be implemented as Streamly.FileSystem.Anonymous module and would have similar APIs as Streamly.Mem.Array.

harendra-kumar commented 5 years ago

What I really mean here is implementing Arrays backed by files on the filesystem. We can even call it Streamly.FileSystem.Array.