search

atteneder / KtxUnity

Load KTX and Basis Universal textures at runtime
Apache License 2.0
223 stars 42 forks source link

Transcoding Basis Files Occasionally Fails on Good Data (with heavy load) #19

Closed sybixsus closed 4 years ago

sybixsus commented 4 years ago

Hi,

So I have an app that loads and transcodes a bunch of BasisU files shortly after a scene loads. Occasionally, the first transcode will fail but the data is good. I think it's happening because I'm initializing a lot of stuff as the new scene starts and I suspect that it's the heavy load on the CPU which is causing the transcode to fail.

I don't have an example at the moment because it's kinda hard to reproduce without a big project. I was wondering if it's related to memory allocation though. I think you're using the Unity Jobs system? Are you using Allocator.TempJob? Because I'm wondering whether maybe I have run into a scenario where the CPU load is such (particularly on mobile) that the job is not complete in four frames. And the memory is being dropped and that's why the transcode fails.

Let me know if there is something I can test or something more I can do to nail down whether this is the problem or if it's something else.

atteneder commented 4 years ago

Hi,

Thanks for reporting! What exactly do you mean when saying "the data is good" ?

A project to reproduce would be helping. Can you give me edge data like number of textures, average size, platform and rough hardware stats you used?

sybixsus commented 4 years ago

When I say "the data is good", I mean that I'm passing the BasisU file as a byte array and that this input is correct. If I l transcode using the same data when the system is not under load, it transcodes correctly.

I will try to come up with a project that reproduces it.

I'm transcoding 16 textures, each 1024x1024. It's on WebGL. Intel i7 6700k 4ghz. 32GB ram, Windows 10. Running from the Unity Editor, so there's a lot more load there too. It's always the first texture that fails and only the first texture, if that helps.

I doubt it's just transcoding images that causes the load. I've just loaded a large scene with complex UI. Lots of other things in my app are also initializing at this time. That's why I think it's going to be hard to come up with a small reproduction project. I can certainly try though.

It might be easier for me to test a fix locally since I already have a project where the problem occurs fairly regularly. Is my theory possible? Do you use Unity Jobs and are you using the Allocator.TempJob? If so, are there changes I can make locally to see whether switching to the permanent allocator fixes the problem?

sybixsus commented 4 years ago

I've tried creating some very stressful stress tests and I'm not able to reproduce. I think it's got to be related to scene loading and perhaps the size or nature of the scene affects the result. But since I'm not able to reproduce this, I don't think it's worth you looking into any further for now.

Feel free to close/delete/whatever. I can always repost if I'm able to deliver a reproduction sample in future.

atteneder commented 4 years ago

Thanks a lot for you effort!

I'd say let's keep it open until the next release (or when it's getting towards a beta/release candidate). Maybe then we reevaluate and eiter fix it or close it.

atteneder commented 4 years ago

Version 0.8.0 is out. Now's the time to reevaluate.

atteneder commented 4 years ago

Closing this one now. Please re-open if it still persists.