Model Optimization Tasks

[Ishaan-Datta]

Examples can be found under /examples/Nvidia TRT Clean, Old Example Conversion, Ultralytics Module, Notebook Tutorials, ONNX Runtime, conversion_tools

Stage 1: Optimizing TensorRT Inference

• [ ] TensorRT Loading YoloV8 Example
• [ ] Old Nvidia Guide for Inference
• [ ] New Nvidia Guide for Inference
• [ ] ONNX Example
• [ ] Experiment with FP16 Precision mode, compare and report results
• [ ] Experiment with INT8 Precision mode, (research process for calibrating as well), compare and report results
• [ ] Quantization Basics
• [ ] Quantization Blog
• [ ] Quantization Blog 2
• [ ] Quantization Blog 3
• [ ] Quantization Blog 4
• [ ] Knowledge Distillation Blog
• [ ] Knowledge Distillation Blog 2
• [ ] Batched NMS Blog
• [ ] Batched NMS Blog 2
• [ ] Batched NMS Blog 3
• [ ] Batched NMS Blog 4
• [ ] Batched NMS Blog 5

Stage 2: Optimizing Model Conversion

• [ ] Finish the code (/conversion_tools/) for running ONNX models using onnxuntime and include functionality for comparing results with true values
• [ ] Research ONNX GraphSurgeon and how we should use it to optimize model architecture/layers
• [ ] Conversion Blog
• [ ] Explore techniques like test-time augmentation (TTA) for predictions (improving detection accuracy at inference cost)

Stage 3: Optimizing Engine Conversion

• [ ] Work with Kezia for model conversion/inference approaches
• [ ] ONNX to TRT Project
• [ ] Research different engine export methods and benchmark results/create performance report
• [ ] ex. PyTorch -> TensorRT (Torch-TRT or torchscript)
• [ ] ex. PyTorch -> ONNX -> TensorRT
• [ ] ex. Later: TensorFlow -> TensorRT (TF-TRT)
• [ ] ex. Later: Tensorflow -> ONNX -> TensorRT
• [ ] ex. Ultralytics conversion
• [ ] ex. trtexec CLI conversion
• [ ] Research different TensorRT runtimes
• [ ] Research different TensorRT Engine Plugins
• [ ] Investigate GStreamer
• [ ] Investigate DeepStream

Specific Plugins/Extras for Viewing in Drive:

• [ ] crop and resize
• [ ] efficientNMS
• [ ] flattenConcat
• [ ] NMS plugin
• [ ] nvpluginfasterRCNN
• [ ] resizeNearest
• [ ] polygraphy