tringi / win32-iocp-events

Example of waiting for Event Objects by associating them with a I/O Completion Port (IOCP), effectively lifting MAXIMUM_WAIT_OBJECTS (64) limit of WaitForMultipleObjects(Ex) API.
27 stars 3 forks source link

multithreaded calls to AddUnlimitedWaitObject #3

Open phoenicyan opened 2 weeks ago

phoenicyan commented 2 weeks ago

Hello! I started using the UnlimitedWait object, and it worked great until I needed to dynamically add sources of notifications with AddUnlimitedWaitObject(). In one thread I have loop calling WaitUnlimitedWaitEx(), and in another thread trying to call AddUnlimitedWaitObject(). I noticed that sometimes it results in locking for some random time because WaitUnlimitedWaitEx() is not returning after 25 milliseconds. I currently do not understand the nature of the issue, and how it could be fixed. Maybe this is not an issue in the UnlimitedWait, but a bug in my code. Thank you!

I think I made it work (at least cannot reproduce the issue) by adding Sleep at the end of loop:

#define DEFAULT_WAIT    (25)
DWORD WINAPI ThreadProc(LPVOID pParam)
{
    CTraceReader* pThis = (CTraceReader*)pParam;
    while (WaitForSingleObject(pThis->m_hQuit, 0) == WAIT_TIMEOUT)
    {
        const auto WAIT_N = 1;
        ULONG nresults = 0;
        void* results[WAIT_N] = { 0 };
        char tmp_buffer[32 * WAIT_N] = { 0 };
        if (WaitUnlimitedWaitEx(pThis->m_pUnlimitedWait, results, tmp_buffer, WAIT_N, &nresults, DEFAULT_WAIT, TRUE))
            continue;
        Sleep(DEFAULT_WAIT);
    }
    return 0;
}

But I'm not sure why the Sleep made the difference.

tringi commented 2 weeks ago

Hi.

That does seem to be an effect of OS scheduling. The thread looping on WaitUnlimitedWaitEx gets to lock the UnlimitedWait's internals again before the thread trying Add has a chance of succeeding. Adding Sleep makes the scheduler give other threads a chance. Probably even Sleep(0) will be sufficient.

But I'll think of replacing the locking mechanism so it doesn't happen, or happens less.

This whole issue is an effect of the current limitation of WaitUnlimitedWaitEx waiting while its internal structures are locked. I have an idea how to improve it, but it's more complex than I initially thought, so no ETA.

phoenicyan commented 2 weeks ago

I initially tried with Sleep(0), and it improved the situation, but still, from time to time, I noticed a delay. You can see usage of the UnlimitedWait in this repo: github.com/phoenicyan/pgnprofiler.

I will think about a better solution for the issue. Will appreciate if you can bring one as well. Thanks!