Suggestion - Cross Platform Storage Abstractions (StorageFolder, StorageFile) Similar to UWP

[leecow]

@legistek commented on Sat Jun 02 2018

Suggestion - Cross Platform Storage Abstractions (StorageFolder, StorageFile) Similar to UWP

General

UWP has an excellent mechanism for abstracting file and folder storage in its StorageFolder and StorageFile classes, a variation of which I believe would make a fine addition to .NET Core.

For those unfamiliar, UWP largely has done away with the use of file paths and pretty much eliminated the need to use System.IO.File or System.IO.Path. Instead you obtain StorageFolder or StorageFile references in various ways, including through file/folder pickers, from one of the predefined application folders, etc. Streams are obtained using StorageFile.OpenReadAsync and OpenWriteAsync.

While it's not perfect - and the UWP system is very restrictive (by design) - it provides a nice abstraction layer for dealing with different kinds of storage content, such as in the cloud.

In .NET Core a similar storage abstraction mechanism would solve a lot of problems developers face when trying to deal with storage across platforms and across different storage types within the same platform. Imagine all the disparate ways storage is handled in Windows apps vs. websites vs. databases vs. mobile devices, all of which could be brought into uniformity through a StorageFile / StorageFolder framework:

• Cloud storage (e.g., Azure blobs or OneDrive)
• HTTP / FTP
• File transfer services like Aspera
• Compressed archives
• Database storage such as NoSQL or, SQL blobs, or FILESTREAM
• Mobile apps with more locked down environments - UWP, iOS, etc.
• Down the road - local storage / session storage in Blazor or Mono WASM apps

Besides making it much more straightforward for applications to be able to support various kinds of storage - and change between them - more easily, this framework would facilitate extremely simple transfer between files and folders of different types. For example, copying a file from disk to the cloud, or to a NoSQL database, or into a Zip archive would all use the same methods.

My proposal is that .NET Core itself would provide a new System.IO.Storage namespace, and abstract base classes - StorageFile and StorageFolder - specifying the minimum required functionality and some utility methods. For example:

public abstract class StorageObject 
{
    public abstract string Name { get; }
    public abstract Task RenameAsync();
    public abstract Task<StorageProperties> GetPropertiesAsync();   // would retrieve metadata like creation/access dates, size, etc.
    public abstract Task DeleteAsync();
    public abstract Task CopyAsync (StorageFolder destination);
    public abstract Task<IEnumerable<StorageObject>> GetChildrenAsync();
}

public abstract class StorageFolder : StorageObject
{
    public abstract int Files { get; }
    public abstract int Folders { get; }
    public abstract Task<StorageFile> GetFileAsync (string filename, bool createIfNotExists);
    public abstract Task<StorageFolder> GetSubfolderAsync (string subfolderName, bool createIfNotExists);

    public virtual Task<IEnumerable<StorageFile>> FindFilesAsync (string criteria, bool includeSubfolders) { ... } // default implementation would use GetChildrenAsync but be overridable for more efficient implementations
    public virtual Task<IEnumerable<StorageFolder>> FindSubfoldersAsync (string criteria, bool includeSubfolders) { ... } // same
}

public abstract class StorageFile : StorageObject
{
    public abstract Task<Stream> OpenReadAsync(bool shared = false);
    public abstract Task<Stream> OpenWriteAsync(bool shared = false); 
}

In addition to the abstract bases, .NET Core itself would also include simple DiskStorageFile and DiskStorageFolder implementations to wrap the existing path-based System.IO classes and methods.

However, other implementations of StorageFile and StorageFolder would be up to library authors. For example, the Azure Storage SDK might include AzureStorageFile and AzureStorageFolder which implemented StorageFile and StorageFolder, wrapping other classes and methods in that library that already work in a very similar way. Blazor might add LocalStorageFIle/Folder and SessionStorageFile/Folder for use with HTML local and session storage. A Zip library like DotNetZip might add ZipStorageFolder and ZipStorageFile to represent the zip archive and its contents. Etc.

It seems like a uniform object oriented storage framework has been a significant omission from the core of .NET since the beginning. Of course in the early 2000s there were not nearly as many storage options as there are today. Frankly, it's getting a little overwhelming now. I for one found it very difficult to determine the best persistent storage mechanism for our web application, which was initially an Azure Web App. We had a couple of false starts first with SQL Filestream and then with Azure blobs before deciding to move to simple VMs and disk storage. It was a lot of work to go back and forth, because each storage option had completely different APIs. A common storage framework would go a long way toward making the underlying storage technology used by any application more transparent and interchangeable, as it really should be.

Looking forward to comments.

---

@leecow commented on Wed Jun 06 2018

Bumping this over to CoreFX.

[fondencn]

Would be a really great idea, especially when dealing with several apps on multiple windows .net platforms (e.g. UWP, .Net Core via MSIX Installer)

[legistek]

Just wondering if this has been given any consideration or is likely ever to happen.