ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
50.1k stars 16.18k forks source link

New Classes are not shown in training and validation verbose even thought the datastore structure seems correct #8266

Closed Symbadian closed 2 years ago

Symbadian commented 2 years ago

Search before asking

Question

Hi All,

I am new to the object detection and would like to learn my errors in my developmental stages,

Forgive my lack of ignorance but I keep getting this error when I try visualizing the new classes that were added in the training when running the validation script as per your direction python val.py --verbose.

None of the new classes are being shown in the validation verbose stage only the 80 classes per the original dataset. Based on all of the examples provided, I followed such to the best of my knowledge and I cannot seem to get this understanding.

Can you help me analyze this, please? I modified all configurations based on my knowledge but this keeps on popping up, please help!! and I am not sure what this means no matter how much I google??!!??!?

` (base) Symbadian-MacBook-Pro:yolov5 symbadian$ python3 val.py --verbose val: data=data/coco128.yaml, weights=yolov5s.pt, batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.6, task=val, device=, workers=8, single_cls=False, augment=False, verbose=True, save_txt=False, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False, dnn=False YOLOv5 🚀 2022-5-7 torch 1.11.0 CPU

Fusing layers... YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients This is data in the val.py file {'path': '../datasets/coco128', 'train': ['/Users/symbadian/Desktop/LIRIS_new_dataset/datasets/coco128/images/train', '/Users/symbadian/Desktop/LIRIS_new_dataset/datasets/coco128/images/train2017'], 'val': ['/Users/symbadian/Desktop/LIRIS_new_dataset/datasets/coco128/images/val', '/Users/symbadian/Desktop/LIRIS_new_dataset/datasets/coco128/images/train2017'], 'test': None, 'nc': 106, 'names': ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush', 'Beating', 'CuttingInKitchen', 'Discus_Amgst_ppl', 'Discuss_Wr_OnBrd', 'DiscusW_Give_Item', 'Enter_Room', 'Fencing', 'Fighting', 'Gun_wp_Deploy', 'Hand_Shake', 'Knife_Deploy', 'Leaves_Room', 'Make_Answer_Tel', 'Nunchucks', 'Pick_Up_Object', 'Put_Down_Object', 'Put_Item_In_Draw', 'Reading', 'Shooting', 'Stabbing', 'SumoWrestling', 'Typing_On_KeyBrd', 'Unlock_Dr_Leaves', 'Unlock_Rm_Enters', 'Unlock_Rm_Unsucc', 'Walking']} This is nc from val.py file 106 This is ncm from val.py file 80 Traceback (most recent call last): File "/Users/symbadian/Desktop/LIRIS_new_dataset/yolov5/val.py", line 399, in main(opt) File "/Users/symbadian/Desktop/LIRIS_new_dataset/yolov5/val.py", line 372, in main run(*vars(opt)) File "/Users/symbadian/miniforge3/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(args, **kwargs) File "/Users/symbadian/Desktop/LIR Screenshot 2022-06-18 at 18 12 52 IS_new_dataset/yolov5/val.py", line 170, in run assert ncm == nc, f'{weights[0]} ({ncm} classes) trained on different --data than what you passed ({nc} ' TypeError: 'PosixPath' object is not subscriptable Screenshot 2022-06-20 at 11 07 38 `

Additional

My progress is challenging as my knowledge in the area of object detection is limited as a newbie, but my determination to learn is strong. Thank you in advance for acknowledging my digital presence and I am indeed grateful for your guidance!!

github-actions[bot] commented 2 years ago

👋 Hello @MatParr, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

CI CPU testing

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.

noreenanwar commented 2 years ago

facing same issue.I have deleted some classes and now I am using 6 classes.

Symbadian commented 2 years ago

facing the same issue. I have deleted some classes and now I am using 6 classes.

Hi @noreenanwar, did you get through with the 6 classes selected? What was your configuration strategy? "I tried removing all and using just one but that had no impact on the issue" let me know, please?

I've been searching for a community for yolov5 other than here, I'm on the quest to get the full understanding of what is happening rather than a redirected response that adds more confusion to such. Why, when I get this knowledge in the future I would be sure of the challenge and be able to fix this as well as assist someone else!!

Symbadian commented 2 years ago

Hi @noreenanwar, I think I found out!! So what they meant by changing the classes of the labels starting from 80 is in the label.txt file. for e.g 0 0.5877403846153846 0.53125 0.11177884615384616 0.3641826923076923 80 0.5336538461538461 0.5853365384615384 0.08774038461538461 0.1814903846153846 81 0.5276442307692307 0.6225961538461539 0.32091346153846156 0.5661057692307693 0 0.4387019230769231 0.6538461538461539 0.125 0.49038461538461536

I have 2 humans in the image above and two other classes starting from 80. The classes are now being seen in the verbose process in my image of the red arrow above... THE THING IS??!!!?: I have loads of classes and to relabel them correctly in accordance with ROBOLFOW original annotation sequence is a huge task. I am really searching for an easier solution to that to get all of the labels in correctly according to yolov5 coco128.yaml class sequencing.

@glenn-jocher suggested this piece of code after reading some of the issues online: `REPLACE LABLE DETAILS: import os, fnmatch

def findReplace(directory, find, replace, filePattern): for path, dirs, files in os.walk(os.path.abspath(directory)): for filename in fnmatch.filter(files, filePattern): filepath = os.path.join(path, filename) with open(filepath) as f: s = f.read() s = s.replace(find, replace) with open(filepath, "w") as f: f.write(s)

findReplace('','','',"*.txt")`

CHALLENGE: The dataset set annotated in Roboflow has its automated sequencing, HOW do we, or rather HOW would we know what the annotated sequencing would be and if this script is placing the correct class index in connection with the Robolfow class indexing??

WHY? if done incorrectly, the entire training operations would be off rendering the annotation and model processing useless?

CAN A Professional, Please respond here? You guidance is needed!!!!!!!????

noreenanwar commented 2 years ago

I am using city person dataset and I am still facing the same issue.I don't know how to resolve this problem.

Symbadian commented 2 years ago

I am using city person dataset and I am still facing the same issue.I don't know how to resolve this problem.

look at your labels in the train folder in the dataset directory, they are not labeled in sequence with whatever model .yaml file you are using. for example, roboflow label sequence started with 0 to N(the amount of classes you have) amend the class labels from the latter starting from 80 as per the example above.

The class names are now showing up in the verbose but as per my testing when I run detect on an image that represents the class, it is not working.

I have just a few images therein like max 5 and I am not sure if this limited amount of data is the cause! An imbalance dataset can cause some issues. In the motion of adding another class with more data, but the task of altering all of the class labels manually is very painful.

I even tried running the script that was provided by @glenn-jocher but that generates another file rather replace the details in the .txt files referring to the labels.

This is what I tried and this can be wrong, I am willing to learn so I am just trying this and that to see if it works.

` import os, fnmatch

def findReplace(directory, find, replace, filePattern): for path, dirs, files in os.walk(os.path.abspath(directory)): for filename in fnmatch.filter(files, filePattern): filepath = os.path.join(path, filename) with open(filepath) as f: s = f.read() print('This is with open s',s) s = s.replace(find, replace) print('This is replacing the data in s',s)# used to see what is going on but nothing is printed here with open(filepath, "w") as f: f.write(s) save(f)

findReplace('','','',"*.txt")

print(findReplace('</Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/train/labels>','<2 0.5877403846153846 0.53125 0.11177884615384616 0.3641826923076923>','<0 0.5877403846153846 0.53125 0.11177884615384616 0.3641826923076923>',".txt")) findReplace('</Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/train/labels>','<2 0.4387019230769231 0.6538461538461539 0.125 0.49038461538461536>','<0 0.4387019230769231 0.6538461538461539 0.125 0.49038461538461536>',".txt") findReplace('</Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/train/labels>','<1 0.5336538461538461 0.5853365384615384 0.08774038461538461 0.1814903846153846>','<81 0.5336538461538461 0.5853365384615384 0.08774038461538461 0.1814903846153846>',".txt") findReplace('</Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/train/labels>','<0 0.5276442307692307 0.6225961538461539 0.32091346153846156 0.5661057692307693>','<80 0.5276442307692307 0.6225961538461539 0.32091346153846156 0.5661057692307693>',".txt") findReplace('/Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/valid/labels','<1 0.4483173076923077 0.4411057692307692 0.07451923076923077 0.06971153846153846>','<81 0.4483173076923077 0.4411057692307692 0.07451923076923077 0.06971153846153846>',".txt") findReplace('/Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/valid/labels','<2 0.5420673076923077 0.578125 0.13942307692307693 0.33413461538461536>','<0 0.5420673076923077 0.578125 0.13942307692307693 0.33413461538461536>',".txt") findReplace('/Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/valid/labels','<0 0.4987980769230769 0.5961538461538461 0.20072115384615385 0.39783653846153844>','<81 0.4987980769230769 0.5961538461538461 0.20072115384615385 0.39783653846153844>',"*.txt") findReplace('/Users/symbadian/Desktop/LIRIS_new_dataset/ADNew/AllData_Index_Changed/Beating/valid/labels','<2 0.4963942307692308 0.6057692307692307 0.17307692307692307 0.35096153846153844>','<0 0.4963942307692308 0.6057692307692307 `

noreenanwar commented 2 years ago

Classes

nc: 6 # number of classes names: ['ignore regions' ,'pedestrian', 'rider', 'sitting', 'unusual', 'group']

This is in my yaml model and my train model in dataset is also same.

Symbadian commented 2 years ago

Have you tried adding the original classes with these classes?

for example person to toothbrush is the original coco128 dataset consisting of 80 classes labeled as 0-79 and 'Knife_DeployTr' is my class making this 81 classes 0-80.

In the dataset label.txt file, this is what I mean by adjusting your class labels in the .txt file as below;

0 0.5877403846153846 0.53125 0.11177884615384616 0.3641826923076923 80 0.5336538461538461 0.5853365384615384 0.08774038461538461 0.1814903846153846 81 0.5276442307692307 0.6225961538461539 0.32091346153846156 0.5661057692307693 0 0.4387019230769231 0.6538461538461539 0.125 0.49038461538461536

Screenshot 2022-06-21 at 14 28 37

`path: ../datasets/coco128 # dataset root dir train:

val:

test: # test images (optional)

Classes

nc: 81 # number of classes names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush', 'Knife_DeployTr'] `

noreenanwar commented 2 years ago

Thank you so much for your help.But the thing is that I converted city person dataset into coco and I don't know due to which reason it become so complicated .I have already done this with cityscape and it worked well but I got poor PR.So I have decided to use city person and all in vain..

Symbadian commented 2 years ago

Thank you so much for your help.But the thing is that I converted city person dataset into coco and I don't know due to which reason it become so complicated .I have already done this with cityscape and it worked well but I got poor PR.So I have decided to use city person and all in vain..

How did you convert this? I am finding this rather tedious to convert this manually.

noreenanwar commented 2 years ago

I used this website https://chowdera.com/2022/03/202203132214410572.html

Symbadian commented 2 years ago

https://chowdera.com/2022/03/202203132214410572.html

Ok will check this out thanks for the link

Symbadian commented 2 years ago

I used this website https://chowdera.com/2022/03/202203132214410572.html

This won't work as the processing was completed using roboflow! I am now stuck as I tried converting this manually, just a few images to observe the operations. The manual process did work but the amount of data that has to be converted is a huge task. There must be a better way than this. This is where I request input from the pros. and designers

noreenanwar commented 2 years ago

but It worked for me in case of cityscape dataset but poor PR

Symbadian commented 2 years ago

but I worked for me in case of cityscape dataset but poor PR

Not sure how to implement, this. What's poor PR? poor processing?!!?!?!

noreenanwar commented 2 years ago

Precession and recall curve plus mAP (0.0187,0.00682,0.0947)

Symbadian commented 2 years ago

Ahh I see, ok

Precession and recall curve plus mAP (0.0187,0.00682,0.0947)

Ahh I see, ok

Symbadian commented 2 years ago

Precession and recall curve plus mAP (0.0187,0.00682,0.0947)

Hey @noreenanwar, tried it and that didn't work. I have to find another way

assert ncm == nc, f'{weights[0]} ({ncm} classes) trained on different --data than what you passed ({nc} ' \ TypeError: 'PosixPath' object is not subscriptable

noreenanwar commented 2 years ago

Did you try to convert into coco format before?

Symbadian commented 2 years ago

@noreenanwar good morning to ya, I did, had to stop, I've been sitting too long. The lower extremities went numb!

The same challenge exists where the val.py file keeps trigging that error above! I'm not sure what that is or what it is telling me in the simplest understanding. I've seen examples and descriptions posted via discussions with others with @glenn-jocher, but I still don't get it. Thus far what I am certain of is that all of the labels per the classes must be altered starting from 80 as shown in my previous images or else it's futile. Whether that would now show up in the validation as before with the correct labels this time is left to be determined and correct the error above when running python val.py is another story.

Symbadian commented 2 years ago

Screenshot 2022-06-22 at 10 00 41

This is the communication for which I am referring to and None of the labels are modified to allow such operations hence I'm still seeking this aspect of the solution.

and I just saw this link here!?!?!?!?!?! https://github.com/ultralytics/yolov5/issues/74

Symbadian commented 2 years ago

@noreenanwar hey, is it possible to train using multiple yaml files simultaneously?

noreenanwar commented 2 years ago

yes we can do that like this python detect.py --weights yolov5s.pt yolovx.pt --img 640 --conf 0.25 --source data/images/test -- - -class= 16

github-actions[bot] commented 2 years ago

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Access additional Ultralytics ⚡ resources:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

glenn-jocher commented 10 months ago

Hey @noreenanwar,

It seems like you're trying to use the val.py file and encountering an error related to the classes and TypeError: 'PosixPath' object is not subscriptable. This issue is often related to the different classes in the training data compared to what's expected, particularly when using custom datasets.

In addition, it looks like you're asking about training using multiple YAML files simultaneously. While this isn't a common practice, you should be able to achieve this by merging the contents of the YAML files into a single file. It's essential to ensure class labels and other dataset configurations are correctly handled when doing so.

If you're encountering issues with detecting new classes during validation, it's important to verify that your custom dataset's label files are correctly formatted and that your YAML file is configured to include all your custom classes. It's also essential to ensure that your training and validation data are structured properly and that your label files correspond accurately to your dataset.

I recommend checking the format of the label files, updating the classes in your YAML file, and verifying that everything is correctly set up for your custom dataset. Additionally, checking out the GitHub issue you mentioned may provide some helpful insights.

Feel free to check out the YOLOv5 documentation at https://docs.ultralytics.com/yolov5/ for more information on custom datasets and training.

Let me know if you need further assistance with this or have any other questions!