Yolov8 pose label size issue

[palmcorp]

Here is a sample label file: 0 0.643712 0.548266 0.0960109 0.057663 0.595707 0.519435 2 0.646533 0.526217 2 0.682304 0.548261 2 0.691717 0.577098 2 5 0.547212 0.54338 0.0545761 0.0753696 0.519924 0.581065 2 0.539413 0.574652 2 0.554228 0.53937 2 0.5745 0.505696 2

The yamal files show: kpt_shape: [17, 3] nc = 2

YOLO complains with: rain: WARNING ⚠️ /content/datasets/echo-pose/images/train/E8F3E-9_M.jpg: ignoring corrupt image/label: labels require 56 columns each

I count 51? Where are the other coming from?

[pderrenger]

@palmcorp hello! 😊

It looks like you're encountering an issue with the label format for YOLOv8 pose estimation. The warning indicates that each label should have 56 columns, but your labels have fewer.

Here's a breakdown of the expected format:

1. Keypoint Format: For pose estimation, each keypoint typically requires 3 values: x, y, and visibility.
2. Total Columns: With kpt_shape: [17, 3], you have 17 keypoints, each needing 3 values, totaling 51 columns for keypoints alone.
3. Additional Columns: YOLOv8 expects additional columns for class, bounding box (x, y, width, height), and potentially other metadata, which might explain the discrepancy.

To resolve this:

• Ensure each label line includes the class, bounding box, and all keypoints with their visibility.
• Double-check your dataset configuration and ensure it aligns with the expected format.

If the issue persists, please verify that you're using the latest version of the Ultralytics packages. Updates often include bug fixes and improvements.

For more detailed guidance, you can refer to the Ultralytics HUB documentation.

Feel free to reach out if you have more questions! 🚀

[palmcorp]

HI Paula,

Ultralytics and Tourch are refreshed at each run. I’m not clear what you mean by this statement?

“ Double-check your dataset configuration and ensure it aligns with the expected format”

What is the ‘expected format’ . ? Follows some additional data:

I’ll continue to look if you can point me to the area.

Best and thanks.

Paul

Right now each frame has two CVAT skeletons consisting of 4 keypoints each.

Here’s an example from CVAT:

0 0.643712 0.548266 0.0960109 0.057663 0.595707 0.519435 2 0.646533 0.526217 2 0.682304 0.548261 2 0.691717 0.577098 2

5 0.547212 0.54338 0.0545761 0.0753696 0.519924 0.581065 2 0.539413 0.574652 2 0.554228 0.53937 2 0.5745 0.505696 2

Breaking one down”

Class: 5

Confidence: 0.547212

BBox: 0.54338 0.0545761 0.0753696

Keypoints:

0.519924 0.581065 2 0

539413 0.574652 2

554228 0.53937 2

0.5745 0.505696 2

Here’s the script starting each run:

!pip install ultralytics

!pip install --upgrade torch ultralytics

import os, pdb

from ultralytics import YOLO

!pip install --upgrade torch ultralytics hub-sdk

er @.*** Sent: Wednesday, October 16, 2024 7:04 PM To: ultralytics/hub Cc: Paul Petronelli; Mention Subject: Re: [ultralytics/hub] Yolov8 pose label size issue (Issue #886)

@palmcorp https://github.com/palmcorp hello! 😊

It looks like you're encountering an issue with the label format for YOLOv8 pose estimation. The warning indicates that each label should have 56 columns, but your labels have fewer.

Here's a breakdown of the expected format:

1. Keypoint Format: For pose estimation, each keypoint typically requires 3 values: x, y, and visibility.
2. Total Columns: With kpt_shape: [17, 3], you have 17 keypoints, each needing 3 values, totaling 51 columns for keypoints alone.
3. Additional Columns: YOLOv8 expects additional columns for class, bounding box (x, y, width, height), and potentially other metadata, which might explain the discrepancy.

To resolve this:

• Ensure each label line includes the class, bounding box, and all keypoints with their visibility.
• Double-check your dataset configuration and ensure it aligns with the expected format.

If the issue persists, please verify that you're using the latest version of the Ultralytics packages. Updates often include bug fixes and improvements.

For more detailed guidance, you can refer to the Ultralytics HUB documentation https://docs.ultralytics.com/hub/datasets/ .

Feel free to reach out if you have more questions! 🚀

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/hub/issues/886#issuecomment-2418338572 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXH6FXLMJSETGKTPUJ5ED3Z34LFVAVCNFSM6AAAAABQCOHBW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYGMZTQNJXGI . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABXH6FWJUCQBQDEWR32DGODZ34LFVA5CNFSM6AAAAABQCOHBW6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUQETVQY.gif Message ID: @.***>

[pderrenger]

@palmcorp hi Paul,

Thanks for reaching out! Let's clarify the expected format for YOLOv8 pose labels.

Expected Format:

1. Class: The first value in each line is the class ID.
2. Bounding Box: Followed by 4 values representing the bounding box (x_center, y_center, width, height).
3. Keypoints: Each keypoint requires 3 values (x, y, visibility). With kpt_shape: [17, 3], you need 51 values for keypoints.

Example:

For a single object with 17 keypoints, a label line should look like:

class_id x_center y_center width height x1 y1 v1 x2 y2 v2 ... x17 y17 v17

Your Case:

Since you have 4 keypoints per skeleton, ensure each label line includes:

• Class ID
• Bounding box
• 4 keypoints, each with x, y, and visibility (totaling 12 values for keypoints)

Troubleshooting:

• Double-check that each line in your label files matches the expected format.
• Ensure you're using the latest versions of Ultralytics and Torch, as updates may resolve underlying issues.

If you have any more questions or need further assistance, feel free to ask. We're here to help! 😊

[palmcorp]

Hello Paula,

Yolo still disagrees with me. I count 12 columns in the label and yolo asks for 14.

LMK if you see where to look.

Best

Paul

Here is the complete sequence:

YAML FILES:

kpt_shape: [4,3] For both Model and Data yamls

JSON from CVAT:

{"id":915,"image_id":459,"category_id":1,"segmentation":[],"area":4737.308400000001,"bbox":[571.04,486.21,94.86,49.94],"iscrowd":0,

"attributes":{"occluded":false,"track_id":0,"keyframe":true},"keypoints":[571.04,487.53,2,600.58,509.7,2,640.38,486.21,2,665.9,536.15,2],"num_keypoints":4},

Resulting label after conversion using: from ultralytics.data.converter import convert_coco

With:

convert_coco (

    labels_dir=P,

    save_dir=SAVE_DIR,      #  oc,#o_converted/"

    use_segments=False,

    use_keypoints=True,

    cls91to80=True,

    lvis=False,

)

The label for E8F3E-9_M:

0 0.671668 0.586995 0.126533 0.0187935 0.608402 0.577598 2 0.694217 0.584489 2 0.734935 0.596391 2 0.711239 0.593663 2

5 0.535234 0.610418 0.0907717 0.0387717 0.489848 0.629804 2 0.523033 0.622391 2 0.554239 0.606946 2 0.58062 0.591033 2

Label analysis for 2nd skeletons

Class: 5

BoundingBox: 0.535234 0.610418 0.0907717 0.0387717

Keypoints:

.489848 0.629804 2

0.523033 0.622391 2

0.554239 0.606946 2

0.58062 0.591033 2

I see 12 Columns.

At initialization:

using YAML: /content/yaml_files/modelR.yaml

Ultralytics 8.3.15 🚀 Python-3.10.12 torch-2.4.1+cu121 CPU (Intel Xeon 2.20GHz)

engine/trainer: task=pose, mode=train, model=yolov8n-pose.yaml, data=/content/yaml_files/modelR.yaml, epochs=2, time=None, patience=100, batch=16, imgsz=640, save=True, save_period=-1, cache=False, device=None, workers=8, project=None, name=train4, exist_ok=False, pretrained=yolov8n-pose.pt, optimizer=auto, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=10, resume=False, amp=True, fraction=1.0, profile=False, freeze=None, multi_scale=False, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, vid_stride=1, stream_buffer=False, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, embed=None, show=False, save_frames=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, show_boxes=True, line_width=None, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=True, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, bgr=0.0, mosaic=1.0, mixup=0.0, copy_paste=0.0, copy_paste_mode=flip, auto_augment=randaugment, erasing=0.4, crop_fraction=1.0, cfg=None, tracker=botsort.yaml, save_dir=runs/pose/train4

Overriding model.yaml kpt_shape=[17, 3] with kpt_shape=[4, 3]

Overriding model.yaml nc=1 with nc=2

Here’ the YOLO r3wponw3:

train: WARNING ⚠️ /content/datasets/echo-pose/images/train/E8F3E-99_M.jpg: ignoring corrupt image/label: labels require 14 columns each

From: Paula Derrenger @.*** Sent: Wednesday, October 16, 2024 7:04 PM To: ultralytics/hub Cc: Paul Petronelli; Mention Subject: Re: [ultralytics/hub] Yolov8 pose label size issue (Issue #886)

@palmcorp https://github.com/palmcorp hello! 😊

It looks like you're encountering an issue with the label format for YOLOv8 pose estimation. The warning indicates that each label should have 56 columns, but your labels have fewer.

Here's a breakdown of the expected format:

1. Keypoint Format: For pose estimation, each keypoint typically requires 3 values: x, y, and visibility.
2. Total Columns: With kpt_shape: [17, 3], you have 17 keypoints, each needing 3 values, totaling 51 columns for keypoints alone.
3. Additional Columns: YOLOv8 expects additional columns for class, bounding box (x, y, width, height), and potentially other metadata, which might explain the discrepancy.

To resolve this:

• Ensure each label line includes the class, bounding box, and all keypoints with their visibility.
• Double-check your dataset configuration and ensure it aligns with the expected format.

If the issue persists, please verify that you're using the latest version of the Ultralytics packages. Updates often include bug fixes and improvements.

For more detailed guidance, you can refer to the Ultralytics HUB documentation https://docs.ultralytics.com/hub/datasets/ .

Feel free to reach out if you have more questions! 🚀

— Reply to this email directly, view it on GitHub https://github.com/ultralytics/hub/issues/886#issuecomment-2418338572 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXH6FXLMJSETGKTPUJ5ED3Z34LFVAVCNFSM6AAAAABQCOHBW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYGMZTQNJXGI . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABXH6FWJUCQBQDEWR32DGODZ34LFVA5CNFSM6AAAAABQCOHBW6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUQETVQY.gif Message ID: @.***>

[pderrenger]

Hello Paul,

Thanks for providing the details! It seems like the issue might be related to the label format. Let's break it down:

Expected Label Format:

For each object, the label should include:

1. Class ID: 1 column
2. Bounding Box: 4 columns (x_center, y_center, width, height)
3. Keypoints: For kpt_shape: [4, 3], you need 12 columns (4 keypoints, each with x, y, visibility)

This totals to 17 columns, but YOLO is asking for 14. Let's ensure the configuration is correct:

Troubleshooting Steps:

1. Verify Keypoints: Double-check that each keypoint has 3 values (x, y, visibility).
2. Bounding Box Format: Ensure the bounding box values are correctly formatted.
3. Check YAML Files: Confirm that both model and data YAML files are correctly set with kpt_shape: [4, 3].

Code Example:

Here's a sample label line for reference:

class_id x_center y_center width height x1 y1 v1 x2 y2 v2 x3 y3 v3 x4 y4 v4

Additional Tips:

• Ensure your conversion script is correctly mapping the JSON to the expected label format.
• Verify that you're using the latest versions of Ultralytics and Torch to avoid any version-related issues.

If the problem persists, consider revisiting the conversion process or checking for any discrepancies in the dataset configuration.

Feel free to reach out if you need more help! 😊