Closed WangFengtu1996 closed 6 months ago
convert new engine , using dlacore =1
echo "Build DLA loadable for fp16 and int8" mkdir -p data/loadable TRTEXEC=/usr/src/tensorrt/bin/trtexec ${TRTEXEC} --onnx=data/model/yolov5s_trimmed_reshape_tranpose.onnx --verbose --fp16 --saveEngine=data/loadable/yolov5.fp16.fp16chw16in.fp16chw16out.standalone.dla1.bin --inputIOFormats=fp16:chw16 --outputIOFormats=fp16:chw16 --buildDLAStandalone --useDLACore=1 ${TRTEXEC} --minShapes=images:1x3x672x672 --maxShapes=images:1x3x672x672 --optShapes=images:1x3x672x672 --shapes=images:1x3x672x672 --onnx=data/model/yolov5_trimmed_qat.onnx --useDLACore=1 --buildDLAStandalone --saveEngine=data/loadable/yolov5.int8.int8hwc4in.fp16chw16out.standalone.dla1.bin --inputIOFormats=int8:dla_hwc4 --outputIOFormats=fp16:chw16 --int8 --fp16 --calib=data/model/qat2ptq.cache --precisionConstraints=obey --layerPrecisions="/model.24/m.0/Conv":fp16,"/model.24/m.1/Conv":fp16,"/model.24/m.2/Conv":fp16,"/model.23/cv3/conv/Conv":fp16,"/model.23/cv3/act/Sigmoid":fp16,"/model.23/cv3/act/Mul":fp16
run demo using new engine file. The demo named cudla_yolov5_app is a loop to inference engine.
cudla_yolov5_app
LD_LIBRARY_PATH=./src/matx_reformat/build/:$LD_LIBRARY_PATH ./build/cudla_yolov5_app --engine ./data/loadable/yolov5.int8.int8hwc4in.fp16chw16out.standalone.dla1.bin --image ./data/images/image.jpg --backend cudla_int8
using jtop, the dla0 is running. the dla1 is OFF.
jtop
so, how to inference engine in dla core 1?
by set number in cudlaCreateDevice, can slove the problem.
convert new engine , using dlacore =1
run demo using new engine file. The demo named
cudla_yolov5_app
is a loop to inference engine.using![image](https://github.com/NVIDIA-AI-IOT/cuDLA-samples/assets/21277368/f1f3cf32-8d73-41b2-bc5f-2f68720df70f)
jtop
, the dla0 is running. the dla1 is OFF.so, how to inference engine in dla core 1?