Rotation value and segmenation mask

[abhigoku10]

@duanzhiihao thanks for sharing the code base i have few queries

1. In loading the dataset your change / adding the angle to be +-90 based on the bounding box dimension is it only for fisheye camera or any data with rotation
2. There is no rotated anchor values consider in the anchor so you are not considering the orientation values rigth ? then how come we get rotation boxes very accurately
3. In yolobranch class your getting output of 3 * 6 [x,y,w,h,a,conf] where a=angle is also predicted so r u considering angle into ur training as another parameter label , since to apply sigmoid to that angle "angle = torch.sigmoid(raw[..., 4])"
4. Can we apply the same strategy to other yolo version like yolov4/5?
5. Can we apply the same implementation to segmentation architecture so as to obain the segmentation mask of the person

THnaks in advance

[duanzhiihao]

Thank you for your interest. These are all good questions.

1. I'm not sure which operation you are regarding to. For the below line, it only applies only to axis-aligned bounding boxes, e.g., in COCO. https://github.com/duanzhiihao/RAPiD/blob/deb9cc84abf26b1fa158c28aba1cb060cf0de4c0/datasets.py#L83
2. "There is no rotated anchor values consider in the anchor" This is true. We consider the orientation values by letting the network predict the angle to rotate the anchor bbox.
3. Yes
4. Yes. We didn't do that since when we worked on this project there is no yolo v4/v5
5. Yes, but we do not have the segmentation annotation for fisheye images. If we have seg annotation for fisheye images we will definitely do that. We could train segmentation only on rotated COCO but the model performance will be suboptimal.

[abhigoku10]

@duanzhiihao Thanks for the response

1. I shall try to incroporate the methods into the yolov4/yolov5 and send a pull request
2. If we add a tracker to this which one do u suggest since we have to take into consideration both angle and box cords also

THanks in adavance

[duanzhiihao]

By tracker do you mean a video object tracking algorithm? To my best knowledge I don't know if there is any published tracker designed for fisheye geometry (correct me if there is).

In the past we did some simple research on object tracking in fisheye videos. We tried a single object tracker named SiamMask. Given that it is not trained on fisheye images, it works reasonably okay.

[abhigoku10]

@duanzhiihao thanks for the response