How to get features vector of each detection (bbox, class, confidence) ?

[folkien]

Hi,

Is there possible to get from darknet features vector for each YOLOv4 detection (bbox, class, confidence) ? I've like to use it during video processing for my multiple object tracking algorithm.

[shubham-shahh]

you are planning to run YOLO on images or videos? and are you using the python or C code to run your model?

[folkien]

Video. At the moment i use python with wrapper for library file darknet.so

[shubham-shahh]

Do you want to save it to a .txt file or simply print it on the screen?

[folkien]

I need realtime acces to features vector for each detection for each video frame. I need this as numeric vector access in memory. For example in darknet.py there is function which returns detections as list of tuples (class, confidence, bbox).

def detect_image(network, class_names, image, thresh=.5, hier_thresh=.5, nms=.45):
    """
        Returns a list with highest confidence class and their bbox
    """
    pnum = pointer(c_int(0))
    predict_image(network, image)
    detections = get_network_boxes(network, image.w, image.h,
                                   thresh, hier_thresh, None, 0, pnum, 0)
    num = pnum[0]
    if nms:
        do_nms_sort(detections, num, len(class_names), nms)
    predictions = remove_negatives(detections, class_names, num)
    predictions = decode_detection(predictions)
    free_detections(detections, num)
    return sorted(predictions, key=lambda x: x[1])

I need here to return also feature vector bounded with each detection.

[shubham-shahh]

you can access it by accessing the elements of the detection queue

[folkien]

Could you give me example code how to get it?

[shubham-shahh]

when I am printing the first element of the detection queue i am getting

[('dog', '63.29', (280.6484069824219, 187.42095947265625, 80.493896484375, 47.97364044189453))]

the format is "obj name", "confidence", "co-ordinates"

is that what you are expecting?

[folkien]

No. This is not what i need, because I have this now. I need 'features vector' the 'vector' of Net values before classification. Layer of neural network before the last one layer.

[shubham-shahh]

okay gotcha, ill look into it and let you know and by the way if you are using yolo with openCV it is really simple. refer this video https://www.youtube.com/watch?v=9AycYn9gj1U

[folkien]

Thanks, this video looks good. Unfortunately i'm using darknet.so and darknet python wrapper to handle YOLO model. So i need similar functions to get data from inside darknet.

[shubham-shahh]

But, that's what you need right like the output he extracted from 3 different points from the network?

[laplaceson]

Hi @folkien , Do you resolve ths issue, i also have same question. Thanks

[folkien]

@laplaceson @shubham-shahh No, finally i haven't tested https://www.youtube.com/watch?v=9AycYn9gj1U approach with open cv.

[laplaceson]

Thanks , But this is nothing help actually, it only show how to extract bbox but not its corresponding feature vector.

[xuaner11111]

hello everyone, do you solve this problem? get features vector?

[akashAD98]

im also looking for same , i want bbox coordinates without NMS ? is there any way to get it?

[folkien]

im also looking for same , i want bbox coordinates without NMS ? is there any way to get it?

Actually this is simple. You have to disable call

        do_nms_sort(detections, num, len(class_names), nms)

in method detect_image in darknet.py