search

ultralytics / yolov3

YOLOv3 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
10.18k stars 3.44k forks source link

customize classes to 4 #1641

Closed ardeal closed 3 years ago

ardeal commented 3 years ago

Hi,

For YOLOLayer in the network, the output tensor shape is: batch_size X channels X nx X ny In the default yolov3 network, the channels of YOLOLayer is 255. that output is reshaped to: batch_size X na X no nx ny so channels(255) = na(3) * no(85)

however, if we change classes to 4. the output is reshaped to: batch_size X na(3) X no(9) nx ny

so 9 != channels 255. channels = 255 is decided by the network and can't be changed.

if I change the yaml file according the section: https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data, there is a question: the class number of customized dataset is 4 in yaml, however the class number in the output of YOLOLayer is 85. they doesn't match. how to comprehend this question?

if the class number of customized dataset is 4 in yaml, but the class number in the output of YoloLayer is 85. it means that those 4 classes might be incorrectly classified to the class in those 85 classes. For example, the ground truth of input image x is 2, however, the output class number of the network might be 67. Is my understanding correct?

Thanks and Best Regards, Ardeal

glenn-jocher commented 3 years ago

@ardeal YOLOv3/v5 automatically constructs output layers to the number of requested classes. 4 class models will have different output layers than 80 class models.

ardeal commented 3 years ago

@glenn-jocher , Many thanks for your answer!

in original Yolov3, in the output layer, the tensor will be reshaped from: Tensor A1: batch_size X channels(255) X nx X ny to: Tensor B1: batch_size X na(3) X no()85 nx ny

If we change classes to 4. the output is reshaped from: Tensor A2: batch_size X channels(27) X nx X ny to: Tensor B2: batch_size X na(3) X no(9) nx ny

My doubts are: 1) both Tensor A1 and Tensor A2 are decided by the network. If we want to change Tensor A1 and Tensor A2, we have to change the network in advance. is my understanding correct? 2) in master branch of this repository, the network will be automatically changed if we change classes to 4. is my understanding correct? 3) could you please help to tell me where is the code to do the upper changes? 4) as I know, in archive branch, class number of yolo layer is configured in yolov3.cfg file. if we want to change class number, we will have to change yolov3.cfg file at least. however, in master branch, there is no yolov3.cfg file in the repository. this means that your code will automatically change the yolo layer according to class number but not depends on yolov3.cfg file. is my understanding correct?

Thanks and Best Regards, Ardeal

glenn-jocher commented 3 years ago

@ardeal yes of course! YOLO models are constructed from yaml files in yolo.py parse_model(): https://github.com/ultralytics/yolov3/blob/4f2341c0ad0d0cb15dad0f20bc6f4d92380804ee/models/yolo.py#L200

The Detect() module decides the shapes of it's own output convolutions in it's init method: https://github.com/ultralytics/yolov3/blob/4f2341c0ad0d0cb15dad0f20bc6f4d92380804ee/models/yolo.py#L27-L42

ardeal commented 3 years ago

@glenn-jocher , Many thanks!

one more question: if I customized classes to 4 but start the training from your yolov3 pth file which is trained for 80 classes, how will the code handle the different output layers for classes =4 and classes = 80?

glenn-jocher commented 3 years ago

@ardeal only layers with matching names and matching shapes will transfer weights.

glenn-jocher commented 3 years ago

@ardeal additionally the number of transferred layers will print to screen: i.e. transferred 507/532 layers.

ardeal commented 3 years ago

@glenn-jocher , Thank you!

such as, the following is 2 different yolov3 network: layer1, layer2, ,,,,,, layerN,layer80,layeroutput80 layer1, layer2, ,,,,,, layerN,layer4,layeroutput4

layer1 to layerN will be transferred, right? layer80,layeroutput80 will not be transferred, right?

glenn-jocher commented 3 years ago

@ardeal yes exactly. The transfer logic is established here. We transfer 'intersecting' layers (same shape, same name). https://github.com/ultralytics/yolov3/blob/4f2341c0ad0d0cb15dad0f20bc6f4d92380804ee/train.py#L77-L90

ardeal commented 3 years ago

@glenn-jocher Got it. Many thanks!

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ibrahim014 commented 2 years ago

Hi, I want to change classes from 80 to 1, I have change yaml file in data/coco128. Can anyone please let me know do I need make any other change in other files?

glenn-jocher commented 2 years ago

@ibrahim014 you don't need to specify class count anywhere except your data yaml. See Train Custom Data tutorial for details:

YOLOv3 Tutorials

ibrahim014 commented 2 years ago

@glenn-jocher thanks. I have another question regarding anchors, I have one class and its bounding box sizes varies from medium, small and extremely large, should I go with default anchor size or I need to change? Your suggestions will be highly appreciated.

glenn-jocher commented 2 years ago

@ibrahim014 👋 Hello! Thanks for asking about model anchors. YOLOv5 🚀 uses a new Ultralytics algorithm called AutoAnchor for anchor verification and generation before training starts.

Autoanchor will analyse your anchors against your dataset and training settings (like --img-size), and will adjust your anchors as necessary if it determines the original anchors are a poor fit, or if an anchor count was specified in your model.yaml rather than anchor values, i.e. 

# Specify anchor count (per layer)
anchors: 3

# --OR-- Specify anchor values manually
anchors:
  - [10,13, 16,30, 33,23]  # P3/8
  - [30,61, 62,45, 59,119]  # P4/16
  - [116,90, 156,198, 373,326]  # P5/32

When generating new anchors, autoanchor first applies a kmeans function against your dataset labels (scaled to your training --img-size), and uses kmeans centroids as initial conditions for a Genetic Evolution (GE) algorithm. The GE algorithm will evolve all anchors for 1000 generations under default settings, using CIoU loss (same regression loss used during training) combined with Best Possible Recall (BPR) as its fitness function.

Notebook example: Open In Colab Open In Kaggle

AutoAnchor Screenshot

No action is required on your part to use autoanchor. If you would like to force manual anchors for any reason, you can skip autoanchor with the --noautoanchor flag:

python train.py --noautoanchor
ibrahim014 commented 2 years ago

@glenn-jocher Thanks a million. I have trained model on custom dataset. When I am trying to run inference using this command " python detect.py --source data/images --weights yolov3.pt --conf 0.25 I am getting following error Traceback (most recent call last): File "detect.py", line 30, in <module> from models.common import DetectMultiBackend ImportError: cannot import name 'DetectMultiBackend' from 'models.common'

glenn-jocher commented 2 years ago

@ibrahim014 your YOLOv5 code is out of date. To update YOLOv5:

ibrahim014 commented 2 years ago

@glenn-jocher Thanks. I have successfully trained Yolov3 on custom dataset and it is working pretty well. Now I want to change input channels and retrain another model, can you please let me know where we can change # of input channels? For example I trained model with RGB now I have RGBS (red, green, blue, saliency) or with just one channel (grayscale).

glenn-jocher commented 2 years ago

@ibrahim014 you can build models with different input channel counts specified in a model yaml by adding an nc field, i.e. nc: 1, however the dataloader and augmentation functions are expecting 3 channels and will need significant manual updates to the code to train arbitrary channel models.

ibrahim014 commented 2 years ago

@glenn-jocher Is there any way to check Average Precision and Recall? Because I can see after every epoch it shows Precision, Recall, mAP@.5 mAP@.5:.95. I am currently using one class.

glenn-jocher commented 2 years ago

@ibrahim014 if you have one class then mAP is the same as AP. mAP is mean AP over classes.

ibrahim014 commented 2 years ago

@glenn-jocher Thanks a million for your quick response. How can we restart training if for any reason training stops? For example I have trained model for 100 epochs now I want to continue training from last epoch.

glenn-jocher commented 2 years ago

@ibrahim014 👋 Hello! Thanks for asking about resuming training. YOLOv5 🚀 Learning Rate (LR) schedulers follow predefined LR curves for the fixed number of --epochs defined at training start (default=300), and are designed to fall to a minimum LR on the final epoch for best training results. For this reason you can not modify the number of epochs once training has started.

LR Curves

If your training was interrupted for any reason you may continue where you left off using the --resume argument. If your training fully completed, you can start a new training from any model using the --weights argument. Examples:

Resume Single-GPU

You may not change settings when resuming, and no additional arguments other than --resume should be passed, with an optional path to the checkpoint you'd like to resume from. If no checkpoint is passed the most recently updated last.pt in your yolov5/ directory is automatically found and used:

python train.py --resume  # automatically find latest checkpoint (searches yolov5/ directory)
python train.py --resume path/to/last.pt  # specify resume checkpoint

Resume Multi-GPU

Multi-GPU DDP trainings must be resumed with the same GPUs and DDP command, i.e. assuming 8 GPUs:

python -m torch.distributed.run --nproc_per_node 8 train.py --resume  # resume latest checkpoint
python -m torch.distributed.run --nproc_per_node 8 train.py --resume path/to/last.pt  # specify resume checkpoint

Start from Pretrained

If you would like to start training from a fully trained model, use the --weights argument, not the --resume argument:

python train.py --weights path/to/best.pt  # start from pretrained model

Good luck and let us know if you have any other questions!

ibrahim014 commented 2 years ago

@glenn-jocher Thanks. The precision and recall that appear after every epoch completion calculated on which settings? i.e, what confidence score and IoU used to calculate P and R?

glenn-jocher commented 2 years ago

@ibrahim014 val.py computes F1 over 1000 confidences from 0.0 to 0.1, and returns P and R at the maximum F1 confidence.

image

ibrahim014 commented 2 years ago

@glenn-jocher Thanks, can we calculate Precision & Recall with specified IoU & Confidence score? For example I want to calculate P & R at 0.5 IoU and 0.6 confidence score. Also can you please let me know how you plot above figure? My 2nd query is for adding background images. For background images we need to add images in yolov5\coco128\images\train2017? But what about labels?

glenn-jocher commented 2 years ago

@ibrahim014 all curves are plotted automatically by val.py when run directly or upon training completion, no action is required on your part.

You can modify metrics computation as you see fit in val.py, metrics.py etc.

glenn-jocher commented 2 years ago

@ibrahim014 for background image recommendations see Tips for Best Training Results tutorial:

YOLOv5 Tutorials

ibrahim014 commented 2 years ago

@glenn-jocher How can we save predicted bounding box coordinates?

glenn-jocher commented 2 years ago

@ibrahim014 see PyTorch Hub tutorial:

YOLOv5 Tutorials