nvthread : block on wait() infinite

[GoogleCodeExporter]

What steps will reproduce the problem?
1. compile nvtt in static mode
2. link in a dll (a max plugin in my case)
3. call successive context.process
4. close the main program
5. it still wait for ever after the call of ~ThreadPool

What is the expected output? What do you see instead?
Should close all the threads normally

What version of the product are you using? On what operating system?
The trunk version

Solution :
uncomment the win32 test in the file Thread.cpp in the wait function :

void Thread::wait(Thread * threads, uint count)
{
#if NV_OS_WIN32
    // @@ Is there any advantage in doing this?
    // YES ! I ran into the case !!!
    nvDebugCheck(count < MAXIMUM_WAIT_OBJECTS);

    HANDLE * handles = new HANDLE[count];
    for (uint i = 0; i < count; i++) {
        handles[i] = threads->p->thread;
    }

    DWORD result = WaitForMultipleObjects(count, handles, TRUE, INFINITE);

    delete [] handles;
#else
    for (uint i = 0; i < count; i++) {
        threads[i].wait();
    }
#endif
}

Original issue reported on code.google.com by arkeon.w...@gmail.com on 2 May 2012 at 9:04

[GoogleCodeExporter]

I'm not sure how that would make any difference, and it's not entirely clear 
what's the problem exactly. Can you provide some more details about the state 
of the threads? what event are they waiting for? Where is the main thread 
blocked? Does the main thread somehow terminate while the worker threads are 
still running?

Original comment by cast...@gmail.com on 2 May 2012 at 11:46

[GoogleCodeExporter]

All the bitmaps have been converted properly, the process is finished.
Then when I close the main program (Max) it lock on ~ThreadPool Thread::wait()

At this state all NVTT threads should be closed for a while.
Did I miss something in the doc ?

What I do for each bitmap :
nvtt::InputOptions inputOptions;
inputOptions.setTextureLayout(nvtt::TextureType_2D, width, height);
inputOptions.setMipmapData(pBuff, width, height);
inputOptions.setFormat(nvtt::InputFormat_BGRA_8UB);
inputOptions.setAlphaMode(bitmap->HasAlpha() ? nvtt::AlphaMode_Transparency : 
nvtt::AlphaMode_None);
inputOptions.setMaxExtents(params.maxTextureSize);
inputOptions.setMipmapGeneration(true);
inputOptions.setRoundMode(nvtt::RoundMode_ToNearestPowerOfTwo);

nvtt::OutputOptions outputOptions;
outputOptions.setFileName(destFile.c_str());

nvtt::CompressionOptions compressionOptions;
compressionOptions.setQuality(nvtt::Quality_Production);
compressionOptions.setFormat(bitmap->HasAlpha() ? nvtt::Format_DXT1a : 
nvtt::Format_DXT1);
compressionOptions.setTargetDecoder(nvtt::Decoder_D3D9);

nvtt::Context context;
ddsMode = context.process(inputOptions, compressionOptions, outputOptions);
free(pBuff);

Original comment by arkeon.w...@gmail.com on 3 May 2012 at 7:27

[GoogleCodeExporter]

I see, thanks for the clarification. I'll see if I can reproduce the problem, 
in the mean time, it would be very helpful if you could determine what's the 
state of the worker threads when the main thread is deadlocked.

My expectation is that all the worker threads should be waiting on their start 
event:

> s_pool->startEvents[i].wait();

before the main thread starts them as follows:

> start(NULL, NULL);

when that happens, the workers are started and terminate, because their func 
pointer has been set to NULL:

> if (s_pool->func == NULL) {
>     return;
> }

Then the main thread waits for the workers to actually terminate:

> Thread::wait(workers, workerCount);

Looks like the workers are not really terminating. I'm guessing the workers may 
not be waiting for the start event as expected or that there's a race 
condition. 

I've made some changes to the code in order to improve its robustness. They are 
in changelist 1345. Let me know if that helps. Thanks!

Original comment by cast...@gmail.com on 3 May 2012 at 5:07

[GoogleCodeExporter]

Sorry to tell that but the problem persist :/

I'll try to have some more clue in debug mode
Any advice where to put the break points ?

Thanks for your help

Original comment by arkeon.w...@gmail.com on 3 May 2012 at 5:23

[GoogleCodeExporter]

More clue ^^
Every call to context.process() create a thread and return true.

but the thread lock on wait here on Event::wait() :

void ThreadPool::workerFunc(void * arg) {
    uint i = toU32((uintptr_t)arg); // This is OK, because workerCount should always be much smaller than 2^32

    while(true) 
    {
        s_pool->startEvents[i].wait();

------->nv::ThreadFunc * func = loadAcquirePointer(&s_pool->func);

        if (func == NULL) {
            return;
        }

        func(s_pool->arg);

        s_pool->finishEvents[i].post();
    }
}

Original comment by arkeon.w...@gmail.com on 3 May 2012 at 5:34

[GoogleCodeExporter]

The only thing NULL I found here is the s_pool:workers:arg
otherwise everything is correctly set.
s_pool:allIdle is set to 1

Original comment by arkeon.w...@gmail.com on 3 May 2012 at 5:41

[GoogleCodeExporter]

I'm debugging this and cannot figure out what could be going on your side.

Rereading your original description of the problem, you said:

> compile nvtt in static mode

What do you mean? Did you compile nvtt as a static library (.lib)? If so, what 
version of the CRT link against?

If using WaitForMultipleObjects fixues the issue, I guess that's OK, but I'd 
rather understand what's actually going on.

Original comment by cast...@gmail.com on 3 May 2012 at 6:28

[GoogleCodeExporter]

Yes I build as a .lib and removed the NVTT_SHARED preprocessor.
I use VC9 and build with MD flag.

Does the threads should exit without the start(NULL, NULL); call ?

In fact after the start(NULL, NULL); all the thread shown in debug with the 
wait state are correctly removed ^^

but here with i == 0  
for (uint i = 0; i < count; i++) {
        threads[i].wait();
    }

it lock on DWORD status = WaitForSingleObject (p->thread, INFINITE);

WaitForMultipleObjects  look as a hack :/
It's really strange

Original comment by arkeon.w...@gmail.com on 3 May 2012 at 6:43