Object detection

[arthurdouillard]

We currently only support bbox when used for cropping the image, but we don't support bbox for targets, especially when there are several bbox per image.

Should they been passed through y or via attributes of the dataset?

[angelomenezes]

Hello Arthur!

Thanks for this incredibly useful library. I'd like to know if you and @TLESORT have plans to incorporate some benchmarks of the incremental object detection field in your lib. If so, have you guys already decided on how to pass all bboxes and class information to conform with the scenario structure ? I'm implementing some CL strategies for such task and might be able to help you! =)

[arthurdouillard]

Hey!

I think we would need a new kind of taskset, and a new scenario. I'm not super familiar with continual object detection, but I assume it's similar to continual segmentation, where we could see an image containing both past, present, and future classes right?

If yes, we can draw inspiration from this class https://github.com/Continvvm/continuum/blob/master/continuum/scenarios/segmentation.py#L17

[zhaoedf]

Hey!

I think we would need a new kind of taskset, and a new scenario. I'm not super familiar with continual object detection, but I assume it's similar to continual segmentation, where we could see an image containing both past, present, and future classes right?

If yes, we can draw inspiration from this class https://github.com/Continvvm/continuum/blob/master/continuum/scenarios/segmentation.py#L17

if i wanted to implement object detection scenarios in continuum style(i.e. achieving sth like your incremental semantic segmentation scenarios ), is there anything i might be able to exploit in continuum?

[arthurdouillard]

Hum... I'm not very well aware of the scenarios in Continual Object Detection, but if we assume they are like the scenarios in segmentation (namely sequential, disjoint, and overlap):

• I'd fork the segmentation task set (https://github.com/Continvvm/continuum/blob/master/continuum/tasks/segmentation_task_set.py#L13), and change the handling of y, as it would be in this case a list of tuple [(x, y, w, h, c), ...]
• I'd fork the segmentation scenario (https://github.com/Continvvm/continuum/blob/master/continuum/scenarios/segmentation.py#L17), and change it the way we read which class is present by parsing the y list info
• Create a dataset, like COCO for example, which adopts the 6th task type (https://github.com/Continvvm/continuum/blob/7afc9ac870b025036f18b8e09621f95929aea69a/continuum/tasks/base.py#L18) and returns a tuple with [list of images, list of bbox (see first point), list of task id or none]

So I think it's rather straightforward. You could leave the support of bounding_boxes in the TaskSet and base dataset (https://github.com/Continvvm/continuum/blob/7afc9ac870b025036f18b8e09621f95929aea69a/continuum/datasets/base.py#L164) if you want to be able to crop the image, but I don't think it's your intention.

I can help you in that goal if you need help :)