search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.78k stars 2.98k forks source link

Training MobileNet / YOLO models using PASCAL VOC datasets - no good result so far #895

Closed mozcelikors closed 1 year ago

mozcelikors commented 3 years ago

Hi, We have previously trained fruit example (MobileNet-SSD) and run successfuly. We have PASCAL VOC dataset regarding traffic signs (with 600 images) that are obtained from Google Maps. We would like to to make use of this dataset and come up with a working solution with jetson-inference program (on Jetson Xavier NX).

To do the training, we have issued the following command and training completed successfuly ( giving some warnings):

source activate jetson-inference-env
cd jetson-inference/python/training/detection/ssd
python3 train_ssd.py --dataset-type=voc --data=/projects/ADASProcessorSDK/voc0712-data/data/VOCdevkit/GTSRB --model-dir=models/GTSRB --epochs=10

Warnings:

/.../anaconda3/envs/jetson-inference-env/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:100: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/.../anaconda3/envs/jetson-inference-env/lib/python3.6/site-packages/torch/nn/_reduction.py:43: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
  warnings.warn(warning.format(ret))

Then, we ran the model using the following command:

cd /opt/jetson-inference
python3 /usr/bin/detectnet-console.py file://Izmir.mp4 rtp://127.0.0.1:5000 --model=/opt/jetson-inference/final-model/11-01-2021/ssd-mobilenet.onnx --labels=/opt/jetson-inference/final-model/11-01-2021/labels.txt --input-blob=input_0 --output-cvg=scores --output-bbox=boxes

The command runs successfuly and we are able to get an image. However, there is no detection whatsoever. So we would like to ask your opinion about what might be the problem here:

A) There is something wrong with the Pytorch versioning that tampers with out training procedure. B) The model we chose is not suitable for detecting traffic signs, therefore we should use something like YOLOv3 or YOLOv4 instead. C) Our dataset, which comprises of around 600 images are not sufficient, given that for fruit detection 6000 images are used for training. D) We have some configuration problem we are unaware of. E) We are training the model with images that have various sizes, and we are running a 1280x720 video. Might there be any problem with resolution mismatch of some sort?

Considering the given options, we would like some guidance as to what might be our problem here, also would like to learn if it is possible to train YOLO models with our PASCAL VOC dataset in order to come up with a working .onnx model.

Thank you very much in advance.

dusty-nv commented 3 years ago

Hi @mozcelikors , you may need to train it for more epochs. I train it for a minimum of 30 epochs.

When you run detectnet.py, you can try lowering the threshold with the --threshold argument. Or also try it with the C++ version to see if difference (detectnet program).

Also you can try this run_ssd_example.py script to run inferencing with the original PyTorch model (before it is exported to TensorRT).

If you exported your model to ONNX multiple times, also delete the TensorRT *.engine file so that gets refreshed. Otherwise TensorRT will use older model.

mozcelikors commented 3 years ago

Hi, Thanks for answering on such short notice. I actually forgot to add that I did do it for 200 epochs. Modified the command later. I will check if adjusting threshold works and do the other stuff you mentioned, will post the results.

mozcelikors commented 3 years ago

Also, could you please share with us how to do this for YOLO models if possible?

dusty-nv commented 3 years ago

Also, could you please share with us how to do this for YOLO models if possible?

I don't have explicit support for YOLO in this repo, it requires different pre/post-processing. For maintenance/support, I prefer to keep the tutorial with SSD at this time (also due to the number of YOLO variants). In https://github.com/dusty-nv/jetson-inference/issues/800 some folks appear to be trying it for YOLO-v5.

There are also these other projects that support YOLO with TensorRT:

ingbeeedd commented 3 years ago

Then, is this project not going to convert tensorrt?

dusty-nv commented 3 years ago

These projects support YOLO conversion to TensorRT:

YOLO is not supported in this repo, YOLO has been moving too fast to keep up with support/maintenance. Instead I use SSD + PyTorch in this repo.

ingbeeedd commented 3 years ago

Your repo doesn't apply Tensorrt, but it shows really good performance.

dusty-nv commented 3 years ago

Yes, it does use TensorRT. The Python bindings link to the jetson-inference C++ library which uses TensorRT. The imageNet/detectNet/segNet classes all inherit from the tensorNet class, which is where you find the TensorRT stuff:

https://github.com/dusty-nv/jetson-inference/blob/master/c/tensorNet.cpp