Standardizing basic file properties

[Arlodotexe]

We're still missing basic file properties like size and modified dates though, nobody has drafted an API in this area yet. You can extend the existing implementations and implement your own 'file property' interface, but we should really get this spec'd out.

We're going to discuss this on Discord and hash out what this might look like, then return with a rough draft.

This ticket filed for tracking.

Additional information

Pulled from the proposal

Storage properties

"Storage properties" are the additional information about a resource provided by most file systems. It's everything from the name to the "Last modified" date to the Thumbnail, and the majority of them can change / be changed.

In AbstractStorage, we simply copied the c# properties and recreated StorageFileContentProperties, and even though we only added GetMusicPropertiesAsync and made the property nullable, it caused a lot of confusion for everyone else who implemented it.

We'll continue using our strategy of "separate interfaces" for this. However, there are a LOT of storage properties we can add, so as long as we separate them into different interfaces, can safely leave "which" for later and instead figure out the "how".

There are 3 requirements

• The information must be accessible and always up-to-date.
• We must provide the option to notify the user of changes
• We must provide the option to change the value (separate interface, see mutability vs modifiability below)

This gives us 2 options:

1. Events + properties + SetThingAsync methods, on an object retrieved from an async method.
2. Events + GetThingAsync Methods + SetThingAsync methods

After some brainstorming, we have a basic skeleton that serves as a guide for any property set.

// The recommended pattern for file properties. 
public interface IStorageProperties<T> : IDisposable
{
    public T Value { get; }

    public event EventHandler<T> ValueUpdated;
}

// Example usage with music properties.
public class MusicData
{
    // todo
}

public interface IMusicProperties
{
    // Must be behind an async method
    public Task<IStorageProperties<MusicData>> GetMusicPropertiesAsync();
}

// If the implementor is capable, they can implement modifying props as well.
public interface IModifiableMusicProperties
{
    public Task UpdateMusicProperties(MusicData newValue);
}

Doing it this way means

• There's a standard for getting and tracking groups of properties. Handy for property set designers and enables extensibility scenarios.
• You can easily check if a file/folder supports the interface beforehand, with try/catch or unboxing.
• You won't get name collisions between multiple property sets
• It's behind an async task, meaning it can retrieved asynchronously and easily disposed of, allowing for setup/teardown of sockets, file handles, etc..
• There's no standard for updating properties, it's up to whoever designs the interface. Update one thing at a time, or update the entire object- whatever is most appropriate for the situation (e.g. UpdateMusicPropertiesAsync() vs ChangeNameAsync()).

[HEIC-to-JPEG-Dev]

For core properties, like Attributes, Size, etc, I would propsose staying with what people know/expect; either as BasicFileProperties (the UWP way) or via class properties (for example SystemFile.Attributes), the latter being my preference as this is what I would expect if I were coming from WinForms, WPF, WinUI, and most languages. I have discussed this in issue #27

For all other properties, I see 3 ways to implement: (1 & 2 allow for easy property update events as required)

1. As a core property. For example, FileExtension or IsSelected or IsPhotoFile. This is simple and intellisense can use them to assist developers; however, this will get to be a lot of properties unless controlled well. This is very performant.
2. A class within the SystemFile class. For example, classes such as "Metadata", "Attributes", "FileTypes", and called in a namespace style: systemFile.Attributes.IsHidden or systemFile.Metadata.IsSelectedInUI. This allows the properties to be grouped, not clutter the core interface, and still work in intellisense. This is not the greatest performing way to do it as the new sub-classes have to be created and put on the heap for every file.
3. A properties dictionary. Dictionary<string,string> where the key can be any user defined string or come from an enumeration of standards. The enum could use the ones defined already by the WinRT namespace or be defined in the class. Examples would be, key="IsSelectedInUI. In addition, the method to read and write these properties could define the type; for example var x = GetProperty(key, defaultValue), where the defautl value is speciying the type to be returned and a default value if not found, again, saving boiler-plate code and checking. On a performance note, This is very performant.

[Arlodotexe]

@HEIC-to-JPEG-Dev Please note that this is for standardized properties across all implementations, not for any single implementation like SystemFile.

You can get properties today if you use the underlying libray (see here). We're crafting a contract that all implementations can share to supply properties, without type checks or dropping to the underlying library.

I would propsose staying with what people know/expect; either as BasicFileProperties (the UWP way) or via class properties (for example SystemFile.Attributes), the latter being my preference

Property retrieval needs to happen asynchronously, so we'll most likely go with the object approach. See OP for details on how we might do this.

A properties dictionary. Dictionary<string,string> where the key can be any user defined string or come from an enumeration of standards.

We use optional interfaces to indicate support, which reduces the work needed from the implementor and enables a compile-time feature support flag. Since they're split across multiple interfaces, enumeration of all properties is going to be tricky.

Interestingly, Windows.Storage uses a Dictionary<string, object> for the basic values. I don't like having a string as the key and an object as the value, because you lose strong typing.

Enumeration of all properties will require a clever and creative approach, and is best saved for later. We want a strongly typed option to be our primary method of interacting with properties.

@HEIC-to-JPEG-Dev Happy to bring you (and our watchers) up to speed - but we've actually already touched on all these things in our Discord thread for basic file properties here. We've got a first draft and some open questions, let's continue our discussion there :)

[Arlodotexe]

Providing inbox interfaces takes careful consideration, even for things that have existed in other APIs for a long time, like LastWriteTime, LastAccessTime and CreationTime. After creating many different storage implementations where they're both supplied by the underlying implementation and consumed by the application, I believe I'm ready to begin the work creating a standardized inbox interface or set of interfaces for them.

For the most part, the problem of generalizing this can be solved in an application without having an inbox standard by passing a delegate in as a method parameter, like so:

public static async Task CopyToAsync(this IFolder sourceFolder, IModifiableFolder destinationFolder, Func<IStorable, DateTime>? getLastUpdateTime = null, CancellationToken cancellationToken = default)
{
}

Where getLastUpdateTime might be defined by the consuming application to point to the data wherever it is:

await nomadFolder.CopyToAsync(workingFolder, storable => storable switch
{
    ReadOnlyKuboNomadFile nFile => nFile.EventStreamEntries.Where(x => x.Id == nFile.Id).Max(x => x.TimestampUtc) ?? ThrowHelper.ThrowNotSupportedException<DateTime>("Unhandled code path"),
    ReadOnlyKuboNomadFolder nFolder => nFolder.EventStreamEntries.Where(x => x.Id == nFolder.Id).Max(x => x.TimestampUtc) ?? ThrowHelper.ThrowNotSupportedException<DateTime>("Unhandled code path"),
    SystemFolder systemFolder => systemFolder.Info.LastWriteTimeUtc,
    SystemFile systemFile => systemFile.Info.LastWriteTimeUtc,
    _ => throw new ArgumentOutOfRangeException(nameof(storable), storable, null),
}, cancellationToken);

Notably, when the information is available, it should already be exposed or available to the file/folder implementation. This means we could have the implementation provide it via an optional interface instead (but no extension method/default behavior).

I'd like to use this opportunity to also reassess the IStorageProperty<T> pattern in general. It's meant to be a general skeleton for how to do this in any implementation, but hasn't yet been battle-tested by an inbox implementation.

@yoshiask @d2dyno1 Before I proceed, do you have any examples where you've implemented IStorageProperty<T> in your projects?

[0x5bfa]

I think IStoragePropery should derive from IStorable and we should create a new kind in StorableKind: File, Folder, Property.

Standard properties Some properties should be instantly retrieved from storage layer, I plan to create a new service layer to fill value of basic properties with a retrieval level Standard, IStorageEnumerationService after IStorageQueryService done, in Files. Also I plan to create UI items that implements ILocatableStorable, and Standard properties called StandardStorageItem, which shown on layout pages.

Extra properties Majority of properties aren’t required to be fetched in a matter of seconds. In Files, I plan to use these properties only in Details Pane and Properties window, which can show ProgressRing. Each layer has respective property class (even Git item as well to hold git url, current branch name and so on), and in each they have GetProperty() and GetPropertyGroup() if applicable method. PropertyGroup returns StoragePropertiesGroupItem for Properties window details section.

Get notified Win32API and even WinRT API doesn’t support to get developers notified of change. So, I’m not sure we can…

Write We can use Shell Win32API (or WinRT Storage API…)

[Arlodotexe]

Moving the conversation here from Discord, replying to https://github.com/Arlodotexe/OwlCore.Storage/issues/23#issuecomment-2164040797:

@Arlodotexe said:

a fascinating idea.

If we did that, the properties would be yielded during GetItemsAsync, and since they implement IStorable they basically act like files but with metadata instead of a Stream.

I think the only issue with that idea is that we can't standardize type safe file properties in something as generic as a Stream We'd end up back at checking if (storable is ISomeOptionalProperty).

This is something I hadn't fully considered, it certainly is interesting when considered alongside the "everything is a file" philosophy from Linux that this abstraction is capable of mirroring.

Implementations like https://github.com/Arlodotexe/OwlCore.Storage.OpenAL play on this theme, but don't require deviating from the standard IFile / Stream to do so.

I wonder if the same could apply to properties?

As a naive example of this flexibility, one way to generalize this "properties are files too" concept is to yield a folder with a reserved Id (such as "properties") or a custom IStorable (such as IStorableProperty), and to present each property as a separate file/property.

• Each file has an Id, Name and Stream. We could supply inbox constants for Id/Name for identification and display, and the stream would contain the property value itself in raw bytes.
• Since these APIs are already designed to work asynchronously, there's no extra work to do.
• For ease of use, we'd want to still provide an interface such as IMusicProperties that can be called
• This means we can support both type safe properties and generalized iterable properties.

@itsWindows11 said:

If properties were represented as file-like items in IFolder.GetItemsAsync, how would you link them to a single file/folder?

@Arlodotexe said:

Good point, I shouldn't have tried to mix the IStorableProperty idea into this quite yet. Imagine you retrieve this folder via GetPropertiesAsync or something similar on IFile. The properties folder for an IFolder could be yielded normally during item enumeration This is a lot of complexity we've propped up, even if it depends on existing well-tested abstractions. I'm not convinced yet, but it's interesting to think about. I'd like to let the idea sit and return to it a bit later.

[Arlodotexe]

Still no definitive answer on enumerable properties, but we should look at implementing standard interfaces for the most common properties, specifically starting with LastWriteTimeUtc, CreationTimeUtc and LastAccessTimeUtc.