dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.81k stars 4.61k forks source link

Bring Windows.Storage namespace into System.IO #94092

Closed HEIC-to-JPEG-Dev closed 9 months ago

HEIC-to-JPEG-Dev commented 9 months ago

System.IO has great performance, but there are missing parts that only WinRT Windows.Storage can provide - such as

Of course it is possible to use the WinRT versions, but the performance of those is so bad that it can cripple an application.

For example, to enumerate my OneDrive photos folder with 50,000 files take System.IO 0.63s, WinRT takes 1m 17s - getting the dimensions of the photos requires using WinRT because they are sparse files, and that takes 37 minutes.

So, the request is to bring the missing API's for storage into System.IO so that we have performant file access services.

ghost commented 9 months ago

Tagging subscribers to this area: @dotnet/area-system-io See info in area-owners.md if you want to be subscribed.

Issue Details
System.IO has great performance, but there are missing parts that only WinRT Windows.Storage can provide - such as - GetImagePropertiesAsync - GetVideoPropertiesAsync Of course it is possible to use the WinRT versions, but the performance of those is so bad that it can cripple an application. For example, to enumerate my OneDrive photos folder with 50,000 files take System.IO 0.63s, WinRT takes 1m 17s - getting the dimensions of the photos requires using WinRT because they are sparse files, and that takes 37 minutes. So, the request is to bring the missing API's for storage into System.IO so that we have performant file access services.
Author: HEIC-to-JPEG-Dev
Assignees: -
Labels: `area-System.IO`, `tenet-performance`
Milestone: -
KalleOlaviNiemitalo commented 9 months ago

Do you mean you want to get the dimensions of the images, without causing the entire files to be downloaded from OneDrive to the local computer? If so, perhaps you could use the OneDrive API for that.

HEIC-to-JPEG-Dev commented 9 months ago

That assumes OneDrive, there are also other providers such as iCloud; so it has to be the local System.IO API (or WinRT) as the information is help in the files sparse data. WinRT has a function to see what data is stored locally (dimensions, thumbnail, etc.) then get information without causing a download. this can all be done locally without the internet connection.

KalleOlaviNiemitalo commented 9 months ago

Does Windows provide non-WinRT APIs for such a query, so that .NET could call them? Cloud Filter API documentation looks mostly intended for developers of cloud file providers; I don't see any functions with which an application could query image dimensions or such.

Perhaps the information is available via the Shell API (IShellItem etc.) somehow. If it is, will that be any faster than the WinRT APIs?

HEIC-to-JPEG-Dev commented 9 months ago

So I did some performance work on calling the WinRT disk IO. I used empty projects, one WinUI, the other UWP, the exact same code in both returns massively different performance results. WinUI took 3 minutes vs UWP took 14 seconds. Normal system.IO in WinUI massively outperforms WinRT, but when you need WinRT calls, it hits a wall. This is why I want the WinRT call brought over to native .Net - for my project I have to go back to UWP and I miss C#11 and .Net 7 :(

jkotas commented 9 months ago

It sounds like a performance issue in WinUI that should be fixed in WinUI. Could you please report it in https://github.com/microsoft/cswinrt or https://github.com/microsoft/microsoft-ui-xaml repos instead?

It does not make sense to introduce app-model specific APIs in .NET BCL to work-around performance issues elsewhere.

adamsitnik commented 9 months ago

It does not make sense to introduce app-model specific APIs in .NET BCL to work-around performance issues elsewhere.

I agree with @jkotas, having said that I am going to close the issue.