YOLOX

[microchila]

Search before asking

• [X] I have searched the Yolo Tracking issues and found no similar bug report.

Question

E:\boxmot0831\boxmot\tracking\val.py:194: UserWarning: loadtxt: input contained no data: "E:\boxmot0831\boxmot\runs\dets_n_embs\yolox_m\dets\MOT17-02-FRCNN.txt" dets = np.loadtxt(args.dets_file_path, skiprows=1) E:\boxmot0831\boxmot\tracking\val.py:195: UserWarning: loadtxt: input contained no data: "E:\boxmot0831\boxmot\runs\dets_n_embs\yolox_m\embs\osnet_x0_5_dukemtmcreid\MOT17-02-FRCNN.txt" embs = np.loadtxt(args.embs_file_path) Traceback (most recent call last): File "E:\boxmot0831\boxmot\tracking\val.py", line 438, in run_all(opt) File "E:\boxmot0831\boxmot\tracking\val.py", line 365, in run_all run_generate_mot_results(opt) File "E:\boxmot0831\boxmot\tracking\val.py", line 336, in run_generate_mot_results generate_mot_results(opt, evolve_config) File "E:\boxmot0831\boxmot\tracking\val.py", line 197, in generate_mot_results dets_n_embs = np.concatenate([dets, embs], axis=1) numpy.exceptions.AxisError: axis 1 is out of bounds for array of dimension 1

[microchila]

I find that there is no detection box to write to a txt text file when using the yolox series of models

[mikel-brostrom]

This functionality broke with the last update but I am not planing to support YoloX in the future. Feel free to drop a PR for this. I will gladly review it 😄

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs. Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

[utsavnandi]

This functionality broke with the last update but I am not planing to support YoloX in the future. Feel free to drop a PR for this. I will gladly review it 😄

Would you be able to point me to the place which causes yolox support to break ? If it is minimal, I am willing to work and fix this with a PR. Thanks!

[mikel-brostrom]

Would it be enough for you @utsavnandi to have something similar to https://github.com/mikel-brostrom/boxmot/blob/master/examples/efficient_det_boxmot.ipynb or integrated in the repo? The issue here is that poetry cannot resolve all the dependencies for the different detectors. That is the reason for breaking them out into notebooks

[mikel-brostrom]

This functionality broke with the last update but I am not planing to support YoloX in the future. Feel free to drop a PR for this. I will gladly review it 😄

Would you be able to point me to the place which causes yolox support to break ? If it is minimal, I am willing to work and fix this with a PR. Thanks!

https://github.com/mikel-brostrom/boxmot/blob/master/examples/det/yolox_boxmot.ipynb

[utsavnandi]

These are awesome, I will check these notebooks, thanks! If poetry is the reason why this is breaking, I am not sure how to proceed.

[Fleyderer]

Ok, now I am here too...

I don't think there is anything connected to poetry, while there could be another ultralytics update which is breaking everyone's libraries and functions (This has never happened before and here it is again).

YOLOX model is still returning detections, but after NMS they are filtered out. So I believe there is something in preprocessing or postprocessing, I'll try to debug it.

And I would like to add that it is better to support YOLOX, which gives best metrics on benchmarks, than Ultralytics YOLO, while their "Ulta new super cool" YOLO11, for example, has much worse metrics in people detection and tracking, compared to their YOLOv8 (both are pretrained ofc).

[mikel-brostrom]

YOLOXs ONNX version collides with Ultralytics' for example. That is why I broke down the detections with different frameworks into notebooks. I am just tired of handling the colliding dependencies of different detectors. I figured out it is not sustainable to handles all the dependencies.

[Fleyderer]

Good evening, namesake :smiley:

I've found solution, it is quite specific, so tomorrow I will check my fixes again, maybe refactor them and send PR. So, please, reopen this Issue

And yes, problem was completely in preprocessing and postprocessing functions. Now everything is good and i think will be good in future until next ultralytics update :smiley:

[Fleyderer]

I have a lot to say after one day of debugging this hell zoo of preprocessing functions.

Detectors are ByteTrack pretrained models, which are being used in BoxMOT project too.

First: current preprocessing, which is used in BoxMOT YOLOX demo notebook is incorrect. IDK, when and where scaleFill option appeared, because I didn't see this option and preprocessing type anywhere else.

Second: ByteTrack (And YOLOX too) actual preprocessing idea is resize to fit into [H, W] and pad with (114, 114, 114) (Just like in Ultralytics preprocessing), but padded image is not in the center, it is in the top-left corner, so pad added to the right and bottom of the image.

Third: ByteTrack uses YOLOX 0.1.0, BoxMOT uses YOLOX 0.3.0, and preprocessing has changed between these versions (They removed normalization with mean/std but left legacy option). So now I have some fun with reproducing old ByteTrack preprocessing, using new YOLOX.

Fourth: YOLOX ByteTrack models input image shape is [1440, 800], not [640, 640] :headstone:

Finally, my current task is to make workaround for all I mentioned above and make a PR. These comments are for future generations of ML-researchers :smiley:

[mikel-brostrom]

scaleFIll comes from here: https://github.com/ultralytics/ultralytics/blob/4453ddab93c390c11c9f8bb8d90a761a4cec3342/ultralytics/data/augment.py#L1475

[Fleyderer]

Ok, I meant scaleFill=True. Yes, this parameter exists, but not used in either Ultralytics YOLO or YOLO-X

[Fleyderer]

I will also leave additional info here:

First image is how YOLO-X ByteTrack detector, which is used in notebook, worked before:

Second image is how it works now:

Most interesting thing for me here is that it worked despite two absolutely different preprocessing functions (Rescale image to 640x640 vs Resize to 1400x800 with bottom/right padding if needed). Yes, it worked bad, but not enough to think that something is going wrong.

I knew that model can be resistant to color shifts, rotations etc -- small augmentations applied during training. But I didn't expect it will normally take 640x640 instead of 1400x800 image and find there something looking good.

And yes, there was BGR problem too 💀

[mikel-brostrom]

The scaling was just to speed inference up. However, I was unaware of the bottom/right padding. Then feeding BRG images, following the opencv standard, is consistent with YOLOX 👍

[mikel-brostrom]

Given that YOLOX uses mosaic during training, the performance consequences of the specific implementation of letterbox'ing should be minimal.

[Fleyderer]

I see that YOLOX model expects preprocessed RGB image:

https://github.com/Megvii-BaseDetection/YOLOX/blob/f00a798c8bf59f43ab557a2f3d566afa831c8887/yolox/data/data_augment.py#L142

While writing it, I decided to check it again and now I understand that swap=(2, 0, 1) is not swap from BGR to RGB, but BGR to GRB :laughing:

So it is neither BGR nor RGB, it is GRB

I agree, that if you pass 640x640 image to the model it will be faster than if you pass 1400x800 🙃 But what about metrics? Today I've evaluated YOLOX pretrained model with wrong preprocessing and 640x640 shape: It gave me around 40 MOTA, while it should be 80. There should be reproducibility of original results and then possible performance tricks, but not vice versa :smiley:

[Fleyderer]

BTW, I made sure the outputs in ByteTrack matched with outputs in BoxMOT, so now preprocessing and postprocessing are completely correct.

My main idea is that if you evaluate metrics on YOLOX (for example), you want to see the same metrics as in original repo. After that, you as a package user make your own decision to speed up inference and drop metrics by changing image size etc.

[mikel-brostrom]

While writing it, I decided to check it again and now I understand that swap=(2, 0, 1) is not swap from BGR to RGB, but BGR to GRB 😆

https://github.com/Megvii-BaseDetection/YOLOX/issues/1492#issuecomment-2476524605

[mikel-brostrom]

It gave me around 40 MOTA, while it should be 80. There should be reproducibility of original results and then possible performance tricks, but not vice versa 😃

Have you tried to run the eval with the original resolution of 1400x800?

[Fleyderer]

Meh, I think I've worked too much, you are completely right about channel order.

When everything will be ok, I will eval with the original resolution. My first goal is to repeat original repo metrics, so I'll make PR right after it

[mikel-brostrom]

My main idea is that if you evaluate metrics on YOLOX (for example), you want to see the same metrics as in original repo. After that, you as a package user make your own decision to speed up inference and drop metrics by changing image size etc.

Recently I decided to abstract away the detector in the trackers' evaluation in the CI by feeding precomputed detections and embeddings. This is because the detectors are not the main thing in this repo anyways and it is the most time consuming part. But I agree it is nice to have. Specially for experimenting with different detectors.

[Fleyderer]

I understand your solution, there is always decision between detector and tracker, and precomputed detections is really good one in cases when we evaluate only trackers between each other.

My current task is to make end-to-end good solution, so I need to train my own detectors, compare them all, and then I will search for the best tracker and hyperparams for it, so I need both good detector and tracker evaluation :smiley:

[mikel-brostrom]

Oooh, nice. Keep me in the loop 😄

[Fleyderer]

I made everything work and evaluated model. And...

For yolox_m model, original metrics are here: MOTA 87.0, IDF1 80.1

My metrics with --conf 0.2 are: MOTA 87.1, IDF1 79.9 (HOTA is 71.2) Without changing --conf: MOTA 86.2, IDF1 80.5 (HOTA is 71.3)

I know, this is so close, but now I wonder why they differ :smiley: Maybe you know what could be wrong.

Command I've used for evaluation:

python tracking/val.py --benchmark MOT17 --yolo-model yolox_m.pth --tracking-method bytetrack --reid-model osnet_x0_25_msmt17.pt --source tracking/val_utils/data/MOT17/train --conf 0.2

I tried --conf 0.2 because I believed to one of your comments that it was in original evaluation, but I didn't see where it is used 😄

Of course, I will check out some hyperparameters from original repo by myself, now that's just first results, which are quite good.

And I see that these results are the best in comparison with your table :thinking: Maybe it is because original preprocessing

[mikel-brostrom]

Could be that the eval was run with some small parameter variation that is not reflected in the final state of the original repo...