Model Optimization Onboarding

[Ishaan-Datta]

System Setup:

• [ ] Dualboot Ubuntu w/ Nvidia Container Toolkit and use dockerfile

Nvidia Video Tutorials:

• [x] Jetson AI Fundamentals - S3E2 - Image Classification Inference - YouTube
• [x] Jetson AI Fundamentals - S3E4 - Object Detection Inference - YouTube
• [x] Jetson AI Fundamentals - S3E5 - Training Object Detection Models - YouTube
• [x] Inference Optimization with NVIDIA TensorRT - YouTube

Documentation:

• [ ] Nvidia Documentation
• [ ] Nvidia Basic Workflow (for making docs)
• [x] Nvidia Guide for Benchmarking/Profiling
• [ ] Creating a guide on model optimization after doing research on how layer fusion operations work and doing some benchmarking
• [x] Comprehensive Guide to TF-TRT background (still useful for pytorch)
• [ ] Python API Documentation

PyTorch Exporting:

• [x] CUDA Graphs

Code Examples:

• [x] ONNX TensorRT
• [x] Nvidia Tutorial Example
• [x] Updated Nvidia Tutorial
• [x] Runtimes
• [x] C++ TensorRT workflow

ROS2 Crash Course:

• [x] ROS2 crash course

Optimization:

• [x] ONNX Graphsurgeon
• [x] ONNX graphsurgeon examples
• [x] Nvidia Object Detection Examples
• [x] Nvidia Example ONNX Deployment
• [x] Nvidia Guide for Optimization
• [ ] Boosting Performance and Utilization with Multi-Instance GPU - YouTube
• [ ] Running Multiple Workloads on a Single A100 GPU - YouTube

Nvidia Libraries:

• [ ] NCCL
• [ ] RAPIDS
• [ ] DALI
• [x] TensorRT
• [x] cuDNN: library to accelerate deep neural network computations
• [ ] cuBLAS
• [x] CUDA Toolkit
• [ ] RAPIDS
• [ ] Support Matrix
• [ ] Gstreamer
• [ ] DeepStream SDK
• [ ] DALI
• [x] NVCC
• [ ] cuRAND
• [ ] cuFFT
• [ ] cuSolver
• [ ] cuSPARSELt: library to accelerate sparse matrix-matrix multiplication
• [ ] cuPTI
• [ ] NVML
• [ ] DLPack
• [ ] Thrust
• [ ] cuTensor