PASCAL VOC XML is incorrect for OBB labels

patel-zeel commented 5 months ago

Describe the bug PASCAL VOC XML export generates incorrect values for Oriented Bounding Boxes (OBB) labels.

To Reproduce

Expected behavior I am not sure if PASCAL VOC XML supports OBB labels.

If it supports:
    It should correctly convert the labels.
else:
    It should first convert the labels to axis-aligned bounding boxes and then get the bounds (just like YOLO export).
    It may show a warning for this implicit conversion.

Screenshots

Labeling in label-studio

PASCAL VOC XML

<?xml version="1.0" encoding="utf-8"?>
<annotation>
<folder>images</folder>
<filename>2af821c3-28.2077.44.png</filename>
<source>
<database>MyDatabase</database>
<annotation>COCO2017</annotation>
<image>flickr</image>
<flickrid>NULL</flickrid>
<annotator>1</annotator>
</source>
<owner>
<flickrid>NULL</flickrid>
<name>Label Studio</name>
</owner>
<size>
<width>1120</width>
<height>1120</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>Airplane</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>514</xmin>
<ymin>141</ymin>
<xmax>613</xmax>
<ymax>193</ymax>
</bndbox>
</object>
<object>
<name>Car</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>819</xmin>
<ymin>317</ymin>
<xmax>918</xmax>
<ymax>379</ymax>
</bndbox>
</object>
</annotation>

Ploting PASCAL VOC and YOLO labels after exports

import matplotlib.pyplot as plt
img = plt.imread("images/2af821c3-28.2077.44.png")

fig, ax = plt.subplots(figsize=(8, 8))
plt.imshow(img)

# <xmin>514</xmin>
# <ymin>141</ymin>
# <xmax>613</xmax>
# <ymax>193</ymax>

x1 = 514
y1 = 141
x2 = 613
y2 = 193

plt.plot([x1, x2], [y1, y1], c='g')
plt.plot([x2, x2], [y1, y2], c='g')
plt.plot([x2, x1], [y2, y2], c='g')
plt.plot([x1, x1], [y1, y2], c='g')

# <xmin>819</xmin>
# <ymin>317</ymin>
# <xmax>918</xmax>
# <ymax>379</ymax>

x1 = 819
y1 = 317
x2 = 918
y2 = 379

plt.plot([x1, x2], [y1, y1], c='b')
plt.plot([x2, x2], [y1, y2], c='b')
plt.plot([x2, x1], [y2, y2], c='b')
plt.plot([x1, x1], [y1, y2], c='b', label='PASCAL VOC XML labels')

### YOLO
# 0 0.5042087542087542 0.14983164983164982 0.08922558922558928 0.047138047138047125
# 1 0.7449494949494948 0.33417508417508407 0.10281727886993011 0.10189583947751818
x1 = (0.5042087542087542 - 0.08922558922558928/2) * 1120
x2 = (0.5042087542087542 + 0.08922558922558928/2) * 1120
y1 = (0.14983164983164982 - 0.047138047138047125/2) * 1120
y2 = (0.14983164983164982 + 0.047138047138047125/2) * 1120

plt.plot([x1, x2], [y1, y1], c='g', linestyle='--', linewidth=5)
plt.plot([x2, x2], [y1, y2], c='g', linestyle='--', linewidth=5)
plt.plot([x2, x1], [y2, y2], c='g', linestyle='--', linewidth=5)
plt.plot([x1, x1], [y1, y2], c='g', linestyle='--', linewidth=5)

x1 = (0.7449494949494948 - 0.10281727886993011/2) * 1120
x2 = (0.7449494949494948 + 0.10281727886993011/2) * 1120
y1 = (0.33417508417508407 - 0.10189583947751818/2) * 1120
y2 = (0.33417508417508407 + 0.10189583947751818/2) * 1120

plt.plot([x1, x2], [y1, y1], c='b', linestyle='--', linewidth=5)
plt.plot([x2, x2], [y1, y2], c='b', linestyle='--', linewidth=5)
plt.plot([x2, x1], [y2, y2], c='b', linestyle='--', linewidth=5)
plt.plot([x1, x1], [y1, y2], c='b', label="YOLO labels", linestyle='--', linewidth=5)

plt.legend()
plt.savefig("tmp.png", dpi=200)

Environment (please complete the following information):

OS: Ubuntu 20.04.6 LTS
Label Studio Version 1.12.1

luforestal commented 5 months ago

Have you found a solution? I have a similar problem with my annotations. They look nice and well centered in LabelStudio but when I plot them in python, they look offset like 100 pixels.

patel-zeel commented 5 months ago

@luforestal Since even YOLO OBB export is not yet implemented (https://github.com/HumanSignal/label-studio-converter/pull/281), we use the following workaround:

Export to CSV format which has the following data:
1. (x, y) - Anchor point (not the center but one of the corners)
2. width and height
3. rotation angle in degrees
4. category
  - Convert it to YOLO OBB format with the following custom script.
```
# @Author: Zeel B Patel, https://patel-zeel.github.io/
import numpy as np
import torch
```

from ultralytics.utils.ops import xywhr2xyxyxyxy from typing import Union

label_map = {"Class A": 0, "Class B": 1}

def label_studio_to_yolo_obb(x1, y1, w, h, r, label) -> Union[np.ndarray, torch.Tensor]: """ Convert from Label studio CSV (x1, y1, w, h, r) to YOLO OBB (x1, y1, x2, y2, x3, y3, x4, y4) format

x1: x-cordinate of one corner of the box, range(0, 100)
y1: y-cordinate of one corner of the box, range(0, 100)
w: width of the box, range(0, 100)
h: height of the box, range(0, 100)
r: rotation angle of the box in degrees
label: label of the box

Returns: Label in YOLO OBB format -> array([label_id, x1, y1, x2, y2, x3, y3, x4, y4])
"""

if isinstance(x1, torch.Tensor):
    handle = torch
    handle.concatenate = torch.cat
    get_array = lambda x: torch.tensor(x)
else:
    handle = np
    get_array = lambda x: np.array(x)

r = handle.radians(r)

cos_rot = handle.cos(r)
sin_rot = handle.sin(r)

x_c = x1 + w / 2 * cos_rot - h / 2 * sin_rot
y_c = y1 + w / 2 * sin_rot + h / 2 * cos_rot

xywhr = get_array([x_c, y_c, w, h, r])
xyxyxyxy = xywhr2xyxyxyxy(xywhr).ravel()

# Normalize to range(0, 1)
xyxyxyxy = xyxyxyxy / 100

label_id = get_array([label_map[label]])
yolo_label = handle.concatenate([label_id, xyxyxyxy])
return yolo_label

* Once we get x1, y1, x2, y2, x3, y3, x4, y4, you can get the bounds and put it inside the XML programatically
```py
xmax = max([x1, x2, x3, x4])
xmin = min([x1, x2, x3, x4])
ymax = max([y1, y2, y3, y4])
ymin = min([y1, y2, y3, y4])

sajarin commented 4 months ago

Thanks for sharing your workaround @patel-zeel, we will update when YOLO OBB export is finally implemented. Otherwise, closing this issue for now!

HumanSignal / label-studio