search

xiongzhu666 / Sky-Segmentation-and-Post-processing

MIT License
110 stars 23 forks source link

CUDA Out of Memory with while running Sky segmentation TRT model Inference on GPU A2000 #9

Closed AnushaManila closed 8 months ago

AnushaManila commented 9 months ago

Hello,

Thanks for sharing your work on Sky Segmentation. I converted the ONNX model to TensorRT and I'm able to run inference on a Video but only for about 90 seconds. Then I see CUDA Runtime Error 2 (Out of Memory)

Environment:

Steps To Reproduce: Step1: Download the ONNX file from Sky-Segmentation and Post processing Git Repo or the link here

Step2: Convert ONNX to TRT using the repo TensorRT-Alpha

Step3: Run inference on a video using the script U2-Net TensorRT

Error Encountered after approx. 90 seconds: image

Any idea what could have caused this?

xiongzhu666 commented 9 months ago

Hello,

Thanks for sharing your work on Sky Segmentation. I converted the ONNX model to TensorRT and I'm able to run inference on a Video but only for about 90 seconds. Then I see CUDA Runtime Error 2 (Out of Memory)

Environment:

  • TensorRT Version: 8.6.1.6
  • NVIDIA GPU: A2000
  • NVIDIA Driver Version: 510.39.01
  • CUDA Version: 11.6
  • CUDNN Version: 8.9.4
  • Operating System: Ubuntu 20.04
  • Python Version : 3.8.10

Steps To Reproduce: Step1: Download the ONNX file from Sky-Segmentation and Post processing Git Repo or the link here

Step2: Convert ONNX to TRT using the repo TensorRT-Alpha

Step3: Run inference on a video using the script U2-Net TensorRT

Error Encountered after approx. 90 seconds: image

Any idea what could have caused this?

I didn‘t deploy by tensorRT，but your speed test is unreasonable. the U2Net inference speed is about 25ms by pytorch and the tensorRT should be more faster. you can refer to other repo and check you tensorRT environment

AnushaManila commented 8 months ago

The inference speed is not the problem, it is realtime with TensorRT. Issue is that the CUDA out of memory happens after running the inference for 90 seconds long.

AnushaManila commented 8 months ago

Found out that Generating the TensorRT Model using this script works all fine Not sure what is wrong with the model generated using /Tensorrt-8.6.1.6/trtexec as in here though Closing this issue