search

mahyarnajibi / SNIPER

SNIPER / AutoFocus is an efficient multi-scale object detection training / inference algorithm
Other
2.69k stars 450 forks source link

thread and multiprocessing in mniterator #155

Closed xws117 closed 5 years ago

xws117 commented 5 years ago

Why use thread_pool in processing image but use multiprocessing.pool in process labels? I tried both use multiprocessing.pool ,that would be slower ,and i don't know whether it is safe...

bharatsingh430 commented 5 years ago

if the data size is more, pool will spawn a process and IPC will be slow defeating the purpose. newer versions of python have shared memory which can get around this issue. thread_pool is faster because it does not start new processes but it only runs on a single core, so its more suitable for io bound operations or some level of parallelism where cpu is not fully utilized