Contribution: Multi-threaded Texture loading

[ghost]

Provided by RafaRafa of the Lost Alpha team. The problem is neither of us can get it to work with your code base properly, hence why I didn't do a pull request.

Here is his code for Lost Alpha: //tatarinrafa:Multithreaded texture loading

u32 bounddelta = 100;       // Main controlling variable. Sets the ammount of textures per thread. Also determines threads count.
                            // if textrures.size = 1900 than number of threads will be 1900 / bounddelta + 1
u32 threads = 0;
u32 readythreadscount = 0;
void TextureLoadingThread(void* p)
{
//  _initialize_cpu_thread();
//  FPU::m64r();
    threads += 1;
    u32 threadnumber = threads;
    u32 upperbound = threadnumber * bounddelta;
    u32 lowerbound = upperbound - bounddelta;
    Msg("Thread# %u is for textures from %u to %u", threadnumber, lowerbound, upperbound);

    for (u32 i = lowerbound; i < upperbound; i++){
        if (i < textures.size()){//prevent empty texture crash
            textures[i]->Load();
            //Msg("Thread# %u: Loading texture# %u with name %s", threadnumber, i, textures[i]->cName.c_str());
        }
    }
    Msg("Thread# %u : Done!", threadnumber);
    readythreadscount += 1;
}

void CResourceManager::DeferredUpload()
{
    if (!RDEVICE.b_is_Ready) return;
    textures.clear();

    for (map_TextureIt t = m_textures.begin(); t != m_textures.end(); t++)
    {
            //t->second->Load();
        textures.push_back(t->second);
    }

    u32 i = (textures.size() / bounddelta) + 1;
    readythreadscount = 0;
    threads = 0;
    Msg("Texture.size = %u, i = %u, delta = %u", textures.size(), i, bounddelta);
    for (u32 y = 1; y <= i; y++){
        Msg("Spawning thread# %u", y);
        //BTHREAD_params                P = { this, V };
        thread_spawn(TextureLoadingThread, "TextureLoadingThread", 0, 0);
    }
    Msg("Spawning threads: Done! readythreadscount = %u", readythreadscount);
    //while (readythreadscount == 0 || readythreadscount != threads) Sleep(100);
    //textures.clear();
    //Msg("DeferredUpload: Done!");
}

Edit Damnit, code thingy broke. But you get the idea.

[ghost]

Doh, and this belongs in ResourceManager.cpp of course. I'm having major brain farts today.

[Im-dex]

@Swartz27 First of all, I'm not sure that CTexture::Load is thread safe. And also I don't see any of threads synchronization mechanism actually you should use. Afaik, Lost Alpha is based on SoC version of xray, so we have no any guarantees that code from there should work properly on CoP.

[ghost]

@Im-dex You're right, I don't know if it's thread safe. In terms of the code, their rendering system is the same as COP, they ported the whole rendering system. Anyway, feel free to discard then.

[ghost]

I forgot to mention: this was tested on the regular COP source code by the creator and it did work, so it works for COP, but since you refactored the way ResourceManager.cpp is we don't know how to get it working. But again, I totally get it if you don't want to use it.

[Im-dex]

This code is invalid due to lack of synchronization. Maybe you are missing something?

[ForserX]

Может я просто с Рафой свяжусь да он сам сюда зайдет да объяснит?

[Xottab-DUTY]

Вставлю свое скромное слово: Почему бы и нет? Будет гораздо лучше, если он сам всё и объяснит.

[ForserX]

Хатаб. Он в целом сказал, что был прирост в скорости загрузки и данный код работал на ванильном ЗП. Суть тут в независимых друг от друга потоках.

[Xottab-DUTY]

Спасибо) Видимо, в независимости их и кроется опасность)