How to use classification-based DL model to update attributes for specific kind of bbox?

[yyyreal]

For example, I have a following task : people @radio=sex:male,female car @radio=type:truck,bus,bicycle

I can use detection-based model to auto annotation people and car in my dataset.

But I wonder how can I use classification-based model to update attributes for specific kind of bounding-box, for example, use gender model to update gender information for people bbox and use another car type model to update car bbox's attributes.

Thanks for your help.

[yyyreal]

Is there anyone can help me? Thx

[bsekachev]

@azhavoro What do you think? Can we do it in current implementation by writing a specific script?

[yyyreal]

@bsekachev Thanks for you reply. I'm really looking forward to this feature. 😀😀

[nmanovic]

@yyyreal ,

Please read the documentation (see attributes below). It should be possible:

# xtl, ytl, xbr, ybr - expected values are float or int
# label - expected value is int
# frame_number - expected value is int
# attributes - dictionary of attribute_name: attribute_value pairs, for example {"confidence": "0.83"}
add_box(self, xtl, ytl, xbr, ybr, label, frame_number, attributes=None)

# points - list of (x, y) pairs of float or int, for example [(57.3, 100), (67, 102.7)]
# label - expected value is int
# frame_number - expected value is int
# attributes - dictionary of attribute_name: attribute_value pairs, for example {"confidence": "0.83"}
add_points(self, points, label, frame_number, attributes=None)
add_polygon(self, points, label, frame_number, attributes=None)
add_polyline(self, points, label, frame_number, attributes=None)

[nmanovic]

@yyyreal ,

Please let me know if it doesn't work for you. I will close the issue for now. Next time please use our gitter channel: https://gitter.im/opencv-cvat

[yyyreal]

@nmanovic hi, I've already read the document you provided, but I don't think it works. Currently the method you provided can only add bounding-box on (for detection task) on the whole frame, but cannot add attributss for specific kind of bbox, just like what I‘ve illustrated in the title.

Maybe the param attributes in function: add_box(self, xtl, ytl, xbr, ybr, label, frame_number, attributes=None) works. But I don't know how to set attributes for the bbox based on the output of a classification-based model automatically.

[nmanovic]

@yyyreal ,

# attributes - dictionary of attribute_name: attribute_value pairs, for example {"confidence": "0.83"}
add_box(self, xtl, ytl, xbr, ybr, label, frame_number, attributes=None)

Did you try to provide attributes argument?

[yyyreal]

My english is not good, but from the following task introduction, you may understand. people @radio=sex:male,female car @radio=type:truck,bus,bicycle

Can the value of the dic attributes comes from a specific DL model. For example I could add gender information to a people. For now I can only use auto-annotation feature to add bbox on my dataset and update attributes for each bbox manually later.

[nmanovic]

@yyyreal ,

You can. Don't ask. Just try.

[yyyreal]

But classification model doesn't come with coordinate xtl, ytl, xbr, ybr

[nmanovic]

@yyyreal ,

CVAT doesn't support classification of images out of the box. But you always can "generate" dummy bounding boxes to classify your images (it is a known workaround at the moment).

[yyyreal]

@nmanovic Definitely, I can use the whole image as a single bounding-box, I've down this work several times. But I really hope you can add this feature, or you can broaden the limit of the anto-annotation feature to support running multiple model in one auto-annotation job, add detection bounding-box and filling the attributes of each bounding-box at the same time.

[nmanovic]

@yyyreal , Now I understand your request. We will create a separate issue for that. I agree that running several models and reuse results of annotations will be useful.

[yyyreal]

@nmanovic Much thanks.😃😃😃