search

lewes6369 / TensorRT-Yolov3

TensorRT for Yolov3
MIT License
489 stars 165 forks source link

Running on DLA - AGX Xavier #86

Open maym86 opened 4 years ago

maym86 commented 4 years ago

I am trying to run yolo on AGX Xavier on the DLA but it looks like the whole model is running on the gpu.

I set the DLA device to 0 here:

https://github.com/lewes6369/TensorRT-Yolov3/blob/b84aa7230830155b21339ed11aa831cec43bef4d/main.cpp#L263

by adding a DLADeivce parameter: net.reset(new trtNet(deployFile, caffemodelFile, outputNames, calibData, run_mode, batchSize, DLADevice));

When I create the engine I get the following which indicates all the layers are still running on the GPU. Has anyone else tried this and have any hints on how to change the model to run on the DLA?


Begin parsing model...
End parsing model...
setFp16Mode
DLA device: 0 of 2
DLA Enabled - Running on DLA core: 0 of 2
Begin building engine...
WARNING: Default DLA is enabled but layer layer1-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer2-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer3-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer4-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer6-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer7-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer8-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer10-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer11-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer13-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer14-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer15-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer17-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer18-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer20-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer21-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer23-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer24-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer26-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer27-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer29-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer30-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer32-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer33-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer35-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer36-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer38-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer39-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer40-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer42-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer43-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer45-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer46-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer48-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer49-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer51-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer52-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer54-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer55-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer57-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer58-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer60-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer61-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer63-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer64-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer65-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer67-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer68-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer70-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer71-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer73-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer74-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer76-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer77-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer78-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer79-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer80-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer81-act is not running on DLA, falling back to GPU.
WARNING: layer84-route: Concatenation on DLA requires at least two inputs.
WARNING: Default DLA is enabled but layer layer84-route is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer85-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer86-upsample is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer88-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer89-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer90-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer91-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer92-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer93-act is not running on DLA, falling back to GPU.
WARNING: layer96-route: Concatenation on DLA requires at least two inputs.
WARNING: Default DLA is enabled but layer layer96-route is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer97-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer98-upsample is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer100-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer101-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer102-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer103-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer104-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer layer105-act is not running on DLA, falling back to GPU.
WARNING: Default DLA is enabled but layer yolo-det is not running on DLA, falling back to GPU.
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer11-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer13-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer14-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer15-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer17-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer18-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer20-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer21-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer23-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer24-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer26-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer27-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer29-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer30-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer32-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer33-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer35-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer36-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer38-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer39-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer40-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer42-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer43-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer45-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer46-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer48-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer49-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer51-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer52-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer54-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer55-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer57-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer58-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer60-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer61-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer63-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer64-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer65-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer67-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer68-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer70-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer71-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer73-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer74-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer76-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer77-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer78-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer79-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer80-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer81-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer85-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer88-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer89-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer90-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer91-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer92-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer93-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer97-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer100-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer101-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer102-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer103-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer104-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer105-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer82-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer94-conv
WARNING: DLA supports only 8 subgraphs per DLA core. Switching to GPU for layer layer106-conv
Collapse```
Roulbac commented 3 years ago

@maym86 Did you manage to solve this?

maym86 commented 3 years ago

Nope. I think the model needs to have specific dla enabled layers which this one didn't

maym86 commented 3 years ago

Docs somewhere probably list which ones can be converted. All I know is for my application having some layers running on dla and some on gpu is very slow and might as well run it all on the gpu.