darknet parallelism with multithreads

[ezequielramos]

I'm having problem using python darknet with multi-threads.

I'm trying to use _loadmeta and _load_net_ just a single time in the main thread and use the loaded values on subthreads to make detections (I'm using darknet.detect_numpy).

I found these code on another branch:

def array_to_image(arr):
    import numpy as np
    # need to return old values to avoid python freeing memory
    arr = arr.transpose(2,0,1)
    c = arr.shape[0]
    h = arr.shape[1]
    w = arr.shape[2]
    arr = np.ascontiguousarray(arr.flat, dtype=np.float32) / 255.0
    data = arr.ctypes.data_as(POINTER(c_float))
    im = IMAGE(w,h,c,data)
    return im, arr

def detect_numpy(net, meta, image, thresh=.5, hier_thresh=.5, nms=.45):
    im, arr = array_to_image(image)
    num = c_int(0)
    pnum = pointer(num)
    predict_image(net, im)
    dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, None, 0, pnum)
    num = pnum[0]
    if (nms): do_nms_obj(dets, num, meta.classes, nms);

    res = []
    for j in range(num):
        for i in range(meta.classes):
            if dets[j].prob[i] > 0:
                b = dets[j].bbox
                res.append((meta.names[i], dets[j].prob[i], (b.x, b.y, b.w, b.h)))
    res = sorted(res, key=lambda x: -x[1])
    free_detections(dets, num)
    return res

So, what I'm trying to do is this:

import darknet
import threading
import time
import scipy.misc

def detectOnThread(meta, net, image, i):

   with open('output_' + str(i) + '.txt','w+') as file:
        res = darknet.detect_numpy(net, meta, image)
        file.write(str(res))

DARKNET_PATH = '<path-to-darknet-folder>'

data = DARKNET_PATH + "/cfg/coco.data"
cfg = DARKNET_PATH + "/cfg/yolov3.cfg"
weight = DARKNET_PATH + "/weights/yolov3.weights"

meta = darknet.load_meta(data)
net = darknet.load_net(cfg, weight, 0)

image = scipy.misc.imread('data/dog.jpg')

threads = []

for i in range(10):
    threads.append(threading.Thread(target=detectOnThread, args=(meta, net, image, i)))
    threads[-1].daemon = True
    threads[-1].start()

for thread in threads:
    while thread.is_alive():
        time.sleep(.01)

But my results on output files generated by detectOnThread function are different and look weird. Some of them are just a empty list, others have detections of aeroplanes and other weird stuff(That aren't detect when running all on main thread).

Isn't really possible to use the same meta and net objets to do multiple simultaneos detections?

Detail: If I send the _load_meta and load_net_ to inside the thread function it works fine(but with lot resource consume).

[hy9be]

Same issue. @ezequielramos Did you figured out the reason behind or any solutions?

@pjreddie Does it mean we are not supposed to use darknet in the way above?

[ezequielramos]

Hi @hy9be . No, I couldn't find the reason behind it.

But I found a "solution" to my problem. Actually, I created a pool of Darknet instances using multiprocessing python module. So I moved the load_meta and load_net to these processes, but like I said before, it consumes much more resource.

[varunsapre]

@ezequielramos Hi! I am trying a similar approach to run multiple instances of darknet using a pool of threads in python (not processes because its easier to share data then). But i keep getting the cuda error: "illegal memory access". I wanted to know if you had any such issues when using multiprocessing instead of multithreading. I believe this is a problem because darknet was not coded to support multiple instances using threads while sharing the same part of GPU RAM. Any help would be appreciated! Thanks

[rebelmachina]

@varunsapre if I use darknet with the python wrapper with Threads and Queue it processes some images, but not all, and it's not deterministic. With Process and Queue it works, but it's much slower.

[ezequielramos]

@whoiscris yes, i got the same results

[carrytameng]

Same issue.

[Rockyzsu]

i used thread lock to fix this issue.