search

yasirkula / UnityNativeGallery

A native Unity plugin to interact with Gallery/Photos on Android & iOS (save and/or load images/videos)
MIT License
1.43k stars 199 forks source link

LoadImageFromPath: Slow and freezes app #195

Open derwaldgeist opened 3 years ago

derwaldgeist commented 3 years ago

So far, I had been reading the image bytes from the gallery file path manually, i.e. using File.ReadAllBytes, loaded them into a texture via texture.LoadImage(), scaled and rotated them using EXIF data and saved the result as a PNG file. This was quite fast, since some of the operations (orientation, scaling) were done in separate threads (async). However, texture.LoadImage() does not support some media formats, e.g. HEIC, so not all gallery files could be used.

Hence, I was very happy to see that there is an own method LoadImagePath that does all of that at once and supports HEIC files, which is awesome. However, I noticed that this method a) takes much longer than my previous routines (sometimes up to 5 secs), and b) it freezes the GUI of the app. In our case, it's an Augmented Reality app, and the whole camera feed freezes while the operation takes place. Which is not a good UX.

It would be awesome if there was an async version of LoadImage() which operates in a separate thread, so at least the freezing problem is gone.

I am also wondering why the operations take so long, especially if I provide a maxWidth. If I don't use this scaling feature (by setting the maxWidth to -1) and scale the texture myself, it is much faster. Here's the script I am using for bilinear scaling, which scales in a separate thread:

http://wiki.unity3d.com/index.php/TextureScale

derwaldgeist commented 3 years ago

Here's the TextureTransform script I am using for orientation changes:

TextureTransform.txt

It's compiled from different StackOverflow answers and I enhanced it so it can be used in combination with this ExifLib (optional):

https://github.com/tedbarnett/read-exif-in-unity

yasirkula commented 3 years ago

I must say that I'm surprised to hear this. I'd also think that maxSize would be faster. I'm not sure if I can optimize its speed, though. The source code is open and I'm open to suggestions. Calling LoadImageAtPath in a separate thread is a possibility but to be honest, currently it is not a priority for me. I want to finish some other projects first 💾 I'll keep this Issue open in the meantime.

derwaldgeist commented 3 years ago

Thanks Yasir. I tried to encapsulate the call in an async Task, but this did not work, because Unity's texture operations are only allowed on the main thread. It throws an Exception otherwise. Likewise, encodeToPNG does not work on a separate thread.

yasirkula commented 3 years ago

Can you run only the following line asynchronously, wait for it to finish and run the rest of the function on the main thread: https://github.com/yasirkula/UnityNativeGallery/blob/0bc6c23925e899e44506f13061287b0f24735957/Plugins/NativeGallery/NativeGallery.cs#L719

Can it be done with Tasks?

HerotechDevs commented 3 years ago

Trying to run that in a task causes the app to crash, I am trying to find some other ways to decode the heic format