search

ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
50.45k stars 16.28k forks source link

validation with val.py fails with a indexing error #7291

Closed lkno0705 closed 2 years ago

lkno0705 commented 2 years ago

Search before asking

YOLOv5 Component

Validation

Bug

I currently have the problem that, if I train a yolo model on a custom dataset, everything works as expected, the model is saved, results get plotted and synced to wandb. But as soon as I try to run the validation seperately with val.py it fails with an index error:

YOLOv5m summary: 369 layers, 21190557 parameters, 0 gradients, 49.1 GFLOPs
WARNING: --img-size 1360 must be multiple of max stride 32, updating to 1376
val: Scanning '/home/leon/studienarbeit/EvalFramework/data/datasets/gtsdb/yolo/val/labels.cache' images and labels..
               Class     Images     Labels          P          R     mAP@.5 mAP@.5:.95:  38%|███▊      | 23/60 [00:0
Traceback (most recent call last):
  File "/home/leon/studienarbeit/EvalFramework/train_yolo.py", line 13, in <module>
    yoloV5.val(dataset=gtsdb_dataset, batch_size=2, weights='yolov5m.pt', img_size=1360)
  File "/home/leon/studienarbeit/EvalFramework/data/models/yolov5/__init__.py", line 25, in val
    yolo_val.run(data=yml, batch_size=batch_size, imgsz=img_size, device=device, weights=weights, task=task)
  File "/home/leon/miniconda3/envs/evalFramework/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/leon/studienarbeit/EvalFramework/data/models/yolov5/yolov5_git/val.py", line 240, in run
    confusion_matrix.process_batch(predn, labelsn)
  File "/home/leon/studienarbeit/EvalFramework/data/models/yolov5/yolov5_git/utils/metrics.py", line 156, in process_batch
    self.matrix[detection_classes[m1[j]], gc] += 1  # correct
IndexError: index 74 is out of bounds for axis 0 with size 44

For reference: the dataset yml:

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: ../../../datasets/gtsdb  # dataset root dir
train: yolo/train/images/  # train images (relative to 'path') 118287 images
test: yolo/test/images/  # 20288 of 40670 images, submit to https://competitions.codalab.org/competitions/20794
val: yolo/val/images/

# Classes
nc: 43  # number of classes
names: [ 'Geschwindigkeitsbegrenzung 20 km/h', 'Geschwindigkeitsbegrenzung 30 km/h', 'Geschwindigkeitsbegrenzung 50 km/h', 'Geschwindigkeitsbegrenzung 60 km/h', 'Geschwindigkeitsbegrenzung 70 km/h', 'Geschwindigkeitsbegrenzung 80 km/h', 'Geschwindigkeitsbegrenzung aufgehoben', 'Geschwindigkeitsbegrenzung 100 km/h', 'Geschwindigkeitsbegrenzung 120 km/h',  'Überholverbot',  'Überholverbot für LKW', 'Vorfahrt',  'Vorfahrtsstraße', 'Vorfahrt gewähren', 'Stopschild',  'Verbot für Fahrzeuge aller Art', 'Verbot für LKW',  'Verbot der Einfahrt',  'Gefahrstelle',  'Kurve links',  'Kurve rechts',  'Doppelkuve',  'Unebene Fahrbahn', 'Schleudergefahr',  'Einseitig verengte Fahrbahn', 'Baustelle', 'Lichtzeichenanlage',  'Fußgängerüberweg', 'spielende Kinder',  'Radfahrer kreuzen', 'Schnee- und Eisglätte', 'Achtung Wildwechsel', 'Alle Streckenverbote aufgehoben', 'Vorgeschriebene Fahrtrichtung rechts',  'Vorgeschriebene Fahrtrichtung links',  'Vorgeschriebene Fahrtrichtung geradeaus', 'Vorgeschriebene Fahrtrichtung geradeaus oder rechts', 'Vorgeschriebene Fahrtrichtung geradeaus oder links', 'Rechts vorbei', 'links vorbei', 'Kreisverkehr',  'Überholverbot aufgehoben',  'Überholverbot für LKW aufgehoben' ]  # class names

I'm calling val.py in a python file as follows:

yml = f"{self.__location__}/yolov5_git/data/{dataset.dataset_id.lower()}.yaml"
batch_size = 2
weights = 'yolov5m.pt'
img_size=1360
device = 0
task = 'val'
yolo_val.run(data=yml, batch_size=batch_size, imgsz=img_size, device=device, weights=weights, task=task)

The prepared dataset looks as follows: Screenshot_20220405_125253 Screenshot_20220405_125356 Screenshot_20220405_125433 with the following structure in every label text file: 4 0.6889705882352941 0.5975 0.01764705882352935 0.030000000000000027 if there are multiple objects in the image, then the next object annotation is written to the next line:

23 0.5422794117647058 0.5549999999999999 0.02132352941176474 0.032500000000000084
2 0.5422794117647058 0.58375 0.015441176470588291 0.02499999999999991
9 0.5426470588235295 0.608125 0.016176470588235348 0.026249999999999996
23 0.32389705882352937 0.551875 0.022794117647058798 0.03374999999999995
2 0.3242647058823529 0.5800000000000001 0.016176470588235292 0.02749999999999997
9 0.32499999999999996 0.60625 0.01470588235294118 0.025000000000000022

The train split includes 720 images, the val and test split include 120 images. I checked all annotations, as this was my first guess but all their classes are correct.

Environment

Minimal Reproducible Example

See bug report above

Additional

Any help would be appreciated! If more information is needed, feel free to ask.

Are you willing to submit a PR?

github-actions[bot] commented 2 years ago

👋 Hello @lkno0705, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

CI CPU testing

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

glenn-jocher commented 2 years ago

@lkno0705 we don't assist in debugging custom code, but you can start from the official usage example shown in val.py. If you encounter any reproducible errors following the official usage example please let us know!

https://github.com/ultralytics/yolov5/blob/5f97001ed4e5deb5c92eb200a79b5cb9da861130/val.py#L5-L18

lkno0705 commented 2 years ago

@glenn-jocher That's understandable. However, the problem also occurs when using the example command in val.py:

python val.py --weights yolov5m.pt --data gtsdb.yaml --img 1360
wandb: Currently logged in as: ***** (use `wandb login --relogin` to force relogin)
val: data=/home/leon/studienarbeit/EvalFramework/data/models/yolov5/yolov5_git/data/gtsdb.yaml, weights=['yolov5m.pt'], batch_size=32, imgsz=1360, conf_thres=0.001, iou_thres=0.6, task=val, device=, workers=8, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False, dnn=False
YOLOv5 🚀 v6.0-392-g0a20c80 torch 1.10.0+cu102 CUDA:0 (NVIDIA GeForce RTX 2080 Ti, 11019MiB)

Fusing layers... 
YOLOv5m summary: 369 layers, 21190557 parameters, 0 gradients, 49.1 GFLOPs
WARNING: --img-size 1360 must be multiple of max stride 32, updating to 1376
val: Scanning '/home/leon/studienarbeit/EvalFramework/data/datasets/gtsdb/yolo/val/labels.cache' images and labels..
               Class     Images     Labels          P          R     mAP@.5 mAP@.5:.95:  25%|██▌       | 1/4 [00:04<
Traceback (most recent call last):
  File "/home/leon/studienarbeit/EvalFramework/val.py", line 390, in <module>
    main(opt)
  File "/home/leon/studienarbeit/EvalFramework/val.py", line 363, in main
    run(**vars(opt))
  File "/home/leon/miniconda3/envs/evalFramework/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/leon/studienarbeit/EvalFramework/val.py", line 240, in run
    confusion_matrix.process_batch(predn, labelsn)
  File "/home/leon/studienarbeit/EvalFramework/data/models/yolov5/yolov5_git/utils/metrics.py", line 156, in process_batch
    self.matrix[detection_classes[m1[j]], gc] += 1  # correct
IndexError: index 74 is out of bounds for axis 0 with size 44
glenn-jocher commented 2 years ago

@lkno0705 thanks for the update! It looks like you are passing incompatible combination of --weights and --data. yolov5m.pt is trained on the COCO dataset, you can not validate it on anything other than the COCO dataset.

lkno0705 commented 2 years ago

@glenn-jocher Yeiks, that mistake was so dumb.. Thanks for the hint! It works now... It makes sense that if you select the correct model to validate, that then the prediction classes are matching and then no index error while validating occurs... Thanks a lot! It seems like I've looked at my screen for too long today! Have a great day!

glenn-jocher commented 2 years ago

@lkno0705 good news 😃! Your original issue may now be improved ✅ in PR #7292. This PR adds better error handling for more informative error messages to help users self-diagnose the problem better.

To receive this update:

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!