Closed blitz-research closed 1 year ago
Haha std :: thread has been replaced by little icons - very useful github!
Closing this as I don't think it's a good idea, rapidobj is designed to be multithreaded so it can load models as fast as possible. I will be switching to tinyobj because I don't need rapidobj's speed, but if I ever do in future I know it's there!
Hi,
I had a quick go at building rapidobj for Emscripten, and got it working with minimal changes! However, my hacks could probably do with some tidying up.
I just reused the __linux #includes and File/FileReader etc class definitons for emscripten which mostly worked except I had to comment out the call to the readahead function. But I'm guessing this largely or even totally negates all the threading going on? Note that emscripten doesn't have access to a 'real' filesystem - it just fakes C I/O on a blob object that gets shipped with emscripten apps, so 'file io' is really just a bunch of synchronous memcpy's. This may change over time though, if a filesystem API for the browser ever gets finalized.
Threads in emscripten are a bit tricky as you have to preallocate them before the app starts via a PTHREAD_POOL_SIZE variable. Plus they're really WebWorkers which ar4e more heavyweight than plain pthreads so you don't want to use too many of them. Rapidobj seems to use std::thread::hardware_concurrency()+5 threads, ie: I had to set PTHREAD_POOL_SIZE to 23 on my 16 thread machine to get it to work. Do you have any idea what this +5 is about? Will it always be '5' everywhere? I guess it could be something going on with esmcripten too, will look into it.
So I think there needs to be some way in rapidobj to at least clamp max thread usage, eg: perhaps replace calls to std::thread::hardware_concurrency() with something like std::min(std::thread::hardware_concurrency(), RAPIDOBJ_MAX_THREADS) - 5
Of course without readahead() tghrading could be pointless, but RAPIDOBJ_MAX_THREADS seems like the easiest way to go unless you're keen on adding something like RAPIDOBJ_SINGLE_THREADED? I had a quick look at doing this but there is a LOT of scary thread code in there!
I also had a quick look at LoadStream in the hope it was single threaded, but that seems to use threads too, not sure how many though, if it's a constant number than it may even be easier to just redirect LoadFile to LoadStream in emscripten.
Anyway, I'll probably create a fork of rapidobj with my emscripten changes, I'll you you know when it's done.
Bye! Mark