Texture FromFile is blocking on the main UI thread

[laultman]

Description

I am loading a Dictionary KVP with the textures from a collection of .png files. This runs as a Task.Run. Inside the task the code paginates the texture into block of files with the idea the app could access textures as soon as the first block loads. The code calls the method stemming from the SK Initialize method. Since Initialize is not async the method calls a method to launch my async code by calling an async void Player method. The Player does some SK initializations and calls an async Task loader to get the textures. The app is using remoting to the HL2. The app starts and the Receiving ... shows in the HL2 It disappears after a few seconds and all is normal. In the debugger all the async code runs to completion as expected. My main menu has a button for a player. When this button is pressed to enable a block of UI instructions. The debugger shows the enable flag is not set which is all the button does. What happens is the HL2 goes blank (the remote stops streaming) for about 10-15 seconds and then resumes streaming. I am at a loss here. The code that is called by the button push is in its own Step having been derived from IStepper.

Platform / Environment

Windows 11 Surface Pro 7 intel i7 16g ram.

Logs or exception details

no errors, no messages on the diagnostics window.

If you have an exception, details about that would also be essential.

[maluoi]

Texture loading is already async, so loading them async is probably not terribly helpful here! In addition to this, accessing certain parameters before they're loaded can cause the application to pause until that content is loaded and accessible.

So like:

SK.Initialize();

string[]   files     = new string[] { "1.png", "2.png", "3.png" };
Material[] materials = files.Select( f => {
    Material result = Material.Default.Copy();
    result[MatParamName.DiffuseTex] = Tex.FromFile(f);
    return result;
});

// DON'T do these here, they will hang the app until colors/width are available
// materials[0][MatParamName.DiffuseTex].GetColors()
// materials[0][MatParamName.DiffuseTex].Width

int curr = 0;
SK.Run(() => {
    int idx = curr % materials.Length;
    curr++;

    // This will draw with "fallback" textures before they've loaded
    Mesh.Cube.Draw(materials[idx], Matrix.T(-1,0,0));

    // This will only draw when all assets are loaded
    if (Assets.CurrentTask >= Assets.TotalTasks) {
        Mesh.Cube.Draw(materials[idx], Matrix.T(0,0,0));
    }

    // This will only draw materials where the texture is fully loaded
    if (material[idx][MatParamName.DiffuseTex].AssetState == AssetState.Loaded) {
        Mesh.Cube.Draw(materials[idx], Matrix.T(1,0,0));
    }

    // This is the correct way to check texture metadata as well
    if (material[idx][MatParamName.DiffuseTex].AssetState >= AssetState.LoadedMeta) {
        Log.Info(""+materials[idx][MatParamName.DiffuseTex].Width);
    }
});

I hope that's helpful?

[laultman]

I've read your docs and knew that you use Async internally but do not understand your implementation. What I just learned from your code is that if you are loading a resource using an SK method, SK is managing the async states internally with respect to the UI thread? Is that correct? (Not just for me to know, there are others who read this stuff). I was under the impression that there was connection to SK's async and my use of Async in my C# code. I could not immediately see how that could be unless you deliberately surfaced it to C# and that didn't seem to fit the overall design of SK. Please correct me if I am wrong in my assumptions. So, when I wrap an async Task.Run around the code in your example 1- above all I have really done is made the call to the SK's loading method a virtually "do nothing" method. I should have been checking the AssetState.Loaded status. That explains very well the observations and the strange behavior of the debugger. Is it safe to say, if I use an SK method to load an asset, just inline it in code, and check status as appropriate for my app? An if I use my own load from a service (disk or cloud) I should implement a similar pattern with a Task.Run() with a return status? Finally, I have not tested this, but does the load actually start on the call to the SK load method, or is it a lazy loader waiting until the first access attempt?

My observations: I have very nearly 500 assets that total about 1gb. The debugger would return from 500 tex.fromfile is about 15ms. That is FAST! I thought my tablet was really screaming! When I subsequently call the method that actually does the drawing in my Update method, the app just stopped showing anything. I mean everything was gone! Crazy thing though - the Step method would still hit a debugger break during that 10-15 second of pause. It was baffling me what was happening - its running, but yeah, no its not? After some time, it would just start up. Naturally after the initial load it was "normal".

Seems some more testing is in order. In case anybody else is interested. This is a flip-frame video player custom implementation for a USMC AI/ML training system. It needs frame-by-frame capabilities. And there are lots of frames.

[laultman]

UPDATE! I implemented the changes suggested. I removed async from all methods wrapping any SK file read write operations and effectively in-lined that code. I do have code that calls our HL2 framework for assets and json data from local and external cloud files and data functions. These are all async .NET libraries and I added a similar AssetState kind-of notification observable to the SK UI thread. This allowed me to place notification messages and the user loading circle function. Much cleaner! Still takes time to load but at least the UI still works.