search

BinomialLLC / crunch

Advanced DXTc texture compression and transcoding library
http://binomial.info
Other
849 stars 183 forks source link

Getting random polka dots on some mipmap levels #31

Open SBRK opened 4 years ago

SBRK commented 4 years ago

Hi, I'm having a weird issue where I get some colored dots added to some mipmap levels of my textures. It appears to be pretty random: those textures were converted automatically by our scripts on a server, but if I try to convert them locally on my computer with the same parameters, I don't have the issue. Additionnaly, some textures have the problem and some don't (for example, the floor has been treated the same way and is fine)

image

SBRK commented 4 years ago

The problem might be linked to parallelisation of treating a lot of files. We convert them to dxt, etc, pvrtc, astc and basis all in parallel. In our logs, we can see basisu failing at the same time one of the dxt file is being treated. Just like the dots, it happens at random and only some basis convertion failed. Maybe the memory runs out?

mredele commented 3 years ago

We're having similar issues. The interesting part is that we've been running the same png to dxt1 (with mipmap) workflow for several years and only very seldom had stripes on some mipmap levels. But recently we changed the machine type (to c5.18xlarge in AWS) consistently got stripes with -helperThreads 8 but I could not reproduce it on my laptop (2018 macbook pro) but with -helperThreads 16 I got similar colored dots as you. Screenshot 2021-01-20 at 17 02 11

mredele commented 3 years ago

The lightmap with a mipmap with dark stripes: https://storage.3d.io/ace/job/2021-01-19/12-04_9c8a5868/lm_1__1.hi-res.gz.dds and the source png: https://storage.3d.io/ace/job/2021-01-19/12-04_9c8a5868/lm_1__1.source.png

richgel999 commented 3 years ago

"The problem might be linked to parallelisation of treating a lot of files. We convert them to dxt, etc, pvrtc, astc and basis all in parallel. In our logs, we can see basisu failing at the same time one of the dxt file is being treated"

Is this issue occurring with basisu.exe, or crunch? The best way (from a perf perspective) to multithread either tool is to disable threading and launch them X times (i.e. running X encodes in parallel). So does the problem go away when you disable threading?

richgel999 commented 3 years ago

Also, note that basisu's threading is implemented in a completely different way vs. crunch. crunch uses completely custom code that leverages the Win32/pthread API's, and basisu uses C++11 std helpers to implement a thread pool with simple jobs.

mredele commented 3 years ago

It's happening with crunch, with only the DXT1 conversion running. It's on Linux (Ubuntu 18.04). -helperThreads 0 seems to fix the problem.

richgel999 commented 3 years ago

Strange - on Linux it uses the pthread API's. It's been many years since I've looked at the crunch codebase, but whatever is happening seems rare. We have some extremely large commercial users of the crunch lib who haven't reported it.

richgel999 commented 3 years ago

There's a decent probability that the issue is in here or the header: https://github.com/BinomialLLC/crunch/blob/master/crnlib/crn_threading_pthreads.cpp