Arlodotexe / OwlCore.Storage

The most flexible file system abstraction, ever. Built in partnership with the UWP Community.
16 stars 4 forks source link

Memory Filesystem #73

Open itsWindows11 opened 2 weeks ago

itsWindows11 commented 2 weeks ago

Since streams in MemoryFiles are made non-disposable as seen in this code: https://github.com/Arlodotexe/OwlCore.Storage/blob/main/src%2FMemory%2FMemoryFile.cs#L52-L61

This can cause memory leaks if not properly managed, so I propose adding a way to control the entire memory filesystem tree, this can be a root MemoryFolder with a list of streams corresponding to virtual paths that's private.

using var virtualMemFilesystem = new MemoryFilesystem();

var memFile = new MemoryFile(fullName: "/virtual/path/file.jpg", filesystem: virtualMemFilesystem); // Can also be created through a parent MemoryFolder that's part of this same filesystem.

var stream = await memFile.OpenWriteAsync();

await GetPhysicalPhotoStream().CopyToAsync(stream);

// For as long as `virtualMemFilesystem` isn't disposed, the files will stay intact in memory.

This opens the door to easily providing virtual filesystems that can be easily disposed or managed based on the app's needs, and may also be used to represent a memory drive in the Windows File Explorer for example which is disposed when unmounted, or when recursive copies are available through this library, provides a way to easily create a filesystem tree for installers to copy all at once.

Arlodotexe commented 2 weeks ago

Note that the internal stream in MemoryFile was passed into the constructor. Here, it's expected that this stream is disposed by the constructor's caller, not internally disposed of by MemoryFile. This allows us to wrap the provided stream with a NonDisposableStreamWrapper and reuse it each time OpenStreamAsync is called, instead of disposing it after the first use and making the IFile instance unusable.

There should be no memory leak here if the caller is properly disposing the provided stream. If there were, we'd want to solve it in a more transparent way than crafting a custom garbage collector for storage / IDisposable.


On the concept of a general Filesystem API, people tend to place here helpers such as file/folder pickers, path tools, etc., and in this case, a memory management system and path aliases.

It's rather vague, and something we try to avoid building inbox. Instead, the storage abstraction focuses heavily on the primitive functionality of what it means to be a "file" or "folder", allowing consumers to leverage the primitives from both the underlying API and the storage abstraction to build more complex helpers and tooling on top of that.