Open glenn-jocher opened 4 years ago
I have a question about the value under column P. Is it map@.60 ? (the default IOU threshold value is .60 at test.py)
I have a question about the value under column P. Is it map@.60 ? (the default IOU threshold value is .60 at test.py)
The default IOU threshold value is NMS threshold, not the map
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Hi, what are those 3 different resolutions that TTA uses? Are they selected randomly? Many thanks
@gizemtanriver they are reduced multiples of the input resolution, they are not fixed values.
@glenn-jocher Thanks! Also, where exactly is this happening in the code? I couldn't find it.
hi everyone. i retrained yolov5 with my custom dataset (2 class) After that, i test my model with input image size (640, 640). i run code with my last.pt model. my output has shape torch.Size([1, 25200, 7]) But the problem is when i run same test with last.onnx. output shape is difference. (3, 80, 80, 7) is that bug? Can you guy help me to solve this problem?
Hi There,
I tried the test.py
with TTA and enlarged the image size to 800 (trained at 640). I checked the saved information which do not look correct. The bounding boxes of label
are all wrong while the pred
images shows the wrong predicted bounding boxes too. When setting img-size back to 640, everything is correct again.
Namespace(augment=True, batch_size=32, conf_thres=0.001, data='dataset/custom.yaml', device='', exist_ok=False, img_size=800, iou_thres=0.6, name='exp', project='runs/test', save_conf=False, save_json=False, save_txt=False, single_cls=False, task='val', verbose=False, weights=['runs/train/exp/weights/best.pt'])
Using torch 1.7.0+cu110 CUDA:0 (GeForce RTX 3090, 24265MB)
Fusing layers...
Model Summary: 316 layers, 21488835 parameters, 0 gradients
Scanning 'dataset/eval/labels/A.cache' for images and labels... 3184 found, 0 missing, 0 empty, 0 corrupted: 100%|█████████████████| 3184/3184 [00:00<?, ?it/s]
Class Images Targets P R mAP@.5 mAP@.5:.95: 100%|██████████████████████████████████████████████████| 106/106 [00:06<00:00, 18.02it/s]
all 3.39e+03 1.59e+04 0.965 0.998 0.995 0.993
Speed: 1.4/0.08/1.5 ms inference/NMS/total per 800x800 image at batch-size 32
In addition, the listed folders or images in the test
option of the data file custom.yaml
seems not be used (from the console output, it said using eval
instead). My dataset is separated in with 3 folders, train, eval and test. e.g.
# train and val data as 1) directory: path/images/, 2) file: path/images.txt, or 3) list: [path1/images/, path2/images/]
train: [dataset/train/images/A,
dataset/train/images/B]
val: [dataset/eval/images/A,
dataset/eval/images/B]
test: [dataset/test/imagesC]
P.S. The best.pt is a yolov5m model.
@ihmc3jn09hk please raise a bug report using the bug report template if you believe there is a reproducible issue, thank you.
@ihmc3jn09hk please raise a bug report using the bug report template if you believe there is a reproducible issue, thank you.
Thank you for your reply. I don't know its related to the parameters I modified, e.g. hyp.scratch.yaml. Will do some more tests.
hi everyone. i retrained yolov5 with my custom dataset (2 class) After that, i test my model with input image size (640, 640). i run code with my last.pt model. my output has shape torch.Size([1, 25200, 7]) But the problem is when i run same test with last.onnx. output shape is difference. (3, 80, 80, 7) is that bug? Can you guy help me to solve this problem?
Hi @HoangTienDuc , i think that you are reading wrong output of the forward. This line below maybe helpful : pred = net.forward(ln)[3]
Hi Glenn,
Why 832? From what I've noticed, you can make the img-size during augmented inference even larger for more accurate results. (In my case, I used img-size = 1664 during inference. The model is version yolo5x and was trained on images at img-size = 640.) Is that correct, or am I overseeing something here? Thanks!
@Shaotran yes you can pass any --img that you want. Results will vary by dataset, among other things. For large image inference you may want to use a P6 model, like yolov5l6.pt or yolov5x6.pt.
Hi, how to use TTA with torch.hub.load(...) ?
@marcusdiy TTA is automatically integrated into all YOLOv5 PyTorch Hub models, and can be accessed by passing augment=True
at inference time. I will update TTA tutorial to make a note of this use case.
results = model(imgs) # inference
results = model(imgs, augment=True) # TTA inference
@glenn-jocher
I would like to ask what 832
in this python val.py --weights yolov5x.pt --data coco.yaml --img 832 --augment --half
is set according to?
@Zengyf-CVer empirical results showed this was the largest that would produce improvements.
Append --augment to any existing val.py command to enable TTA, and increase the image size by about 30% for improved results.
https://github.com/ultralytics/yolov5/pull/10312 pls check my pr - I have fixed this problem
Now it can works like this:
does it work with segmentation?
📚 This guide explains how to use Test Time Augmentation (TTA) during testing and inference for improved mAP and Recall with YOLOv5 🚀. UPDATED 25 September 2022.
Before You Start
Clone repo and install requirements.txt in a Python>=3.7.0 environment, including PyTorch>=1.7. Models and datasets download automatically from the latest YOLOv5 release.
Test Normally
Before trying TTA we want to establish a baseline performance to compare to. This command tests YOLOv5x on COCO val2017 at image size 640 pixels.
yolov5x.pt
is the largest and most accurate model available. Other options areyolov5s.pt
,yolov5m.pt
andyolov5l.pt
, or you own checkpoint from training a custom dataset./weights/best.pt
. For details on all available models please see our README table.Output:
Test with TTA
Append
--augment
to any existingval.py
command to enable TTA, and increase the image size by about 30% for improved results. Note that inference with TTA enabled will typically take about 2-3X the time of normal inference as the images are being left-right flipped and processed at 3 different resolutions, with the outputs merged before NMS. Part of the speed decrease is simply due to larger image sizes (832 vs 640), while part is due to the actual TTA operations.Output:
Inference with TTA
detect.py
TTA inference operates identically toval.py
TTA: simply append--augment
to any existingdetect.py
command:Output:
PyTorch Hub TTA
TTA is automatically integrated into all YOLOv5 PyTorch Hub models, and can be accessed by passing
augment=True
at inference time.Customize
You can customize the TTA ops applied in the YOLOv5
forward_augment()
method here: https://github.com/ultralytics/yolov5/blob/8c6f9e15bfc0000d18b976a95b9d7c17d407ec91/models/yolo.py#L125-L137Environments
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):
Status
If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on MacOS, Windows, and Ubuntu every 24 hours and on every commit.