search

ceccocats / tkDNN

Deep neural network library and toolkit to do high performace inference on NVIDIA jetson platforms
GNU General Public License v2.0
718 stars 209 forks source link

low fps in AGX XAVIER #248

Closed AlanNewImage closed 3 years ago

AlanNewImage commented 3 years ago

Hi:

I test yolo4tiny in AGX use newest repo ,only achieved about 120fps. launched jetson_clocks test by test_rtinference test in two diffrent AGX Xavier and have the same result

Output Buffers: 2 Output Buffers: 2 Output Buffers: 2 Output Buffers: 2 avg: 8.43752

AlanNewImage commented 3 years ago

Output Buffers: 3 Output Buffers: 3 Output Buffers: 3 Output Buffers: 3 avg: 37.9915 this is yolo4_berkely_fp16,only 26fps download weights from this repo, and convert model to .rt by test_yolo4_berkeley

sapienzadavide commented 3 years ago

Hi @AlanNewImage What is the commit/branch you are working on? Thank you

AlanNewImage commented 3 years ago

branch master

I get 244fps for yolo4tiny_fp16 based the newest commit, but only 29fps for yolo4_berkeley_fp16.

sapienzadavide commented 3 years ago

What is the NV power mode? (try MAXN) What is your CUDA/cuDNN/tensorRT version? Thank you.

AlanNewImage commented 3 years ago

Thanks for your reply; CUDA: 10.2 TensorRT: 7.1.3 cudnn: 8.0.0.180

sapienzadavide commented 3 years ago

Ok I tested yolo4tiny_fp16 and yolo4_berkely_pf16 on an AGX xavier with your SW versions (CUDA 10.2, TensorRT 7.1.3, cudnn 8.0.0.180). (commit a992c9feb5fb5c7a64da59f6c5a6f0c1c1a6cf2d) I set the NV power mode to 0 (MAXN), and I ran sudo jetson_clocks.

with test_rtinference: yolo4tiny_fp16 --> ~ 315FPS yolo4_berkely_fp16 --> ~ 40fps

I also tested tkDNN with: NV power mode set to 3 (30W all) (and then ran sudo jetson_clocks again).

yolo4tiny_fp16 --> ~ 210FPS yolo4_berkely_fp16 --> ~ 27fps

Are you sure that NV power mode is set to 0? After changing the power mode, did you run the 'sudo jetson_clocks' command again?

AlanNewImage commented 3 years ago

Thank you very much! I get 335 fps after set nvpmodel to 0 . WOW,it's amazing. Thanks again.