search

isarsoft / yolov4-triton-tensorrt

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
http://www.isarsoft.com
Other
276 stars 63 forks source link

[TRT] Could not compute dimensions. [TRT] Network validation failed. #45

Closed GowthamKudupudi closed 2 years ago

GowthamKudupudi commented 3 years ago

I got a yolov4tiny.wts file. The engine creation fails with this output:

# ./main --network yolov4tiny
[Info] Creating builder
[Info] Creating model yolov4tiny
convBnLeaky: addBatchNorm2d: linx: 0
addBatchNorm2d: lname: module_list.0.BatchNorm2d; shift.count:32;
convBnLeaky: addBatchNorm2d: linx: 1
addBatchNorm2d: lname: module_list.1.BatchNorm2d; shift.count:64;
convBnLeaky: addBatchNorm2d: linx: 2
addBatchNorm2d: lname: module_list.2.BatchNorm2d; shift.count:64;
convBnLeaky: addBatchNorm2d: linx: 4
addBatchNorm2d: lname: module_list.4.BatchNorm2d; shift.count:32;
convBnLeaky: addBatchNorm2d: linx: 5
addBatchNorm2d: lname: module_list.5.BatchNorm2d; shift.count:32;
Adding route 6
Adding leaky convolution 7
convBnLeaky: addBatchNorm2d: linx: 7
addBatchNorm2d: lname: module_list.7.BatchNorm2d; shift.count:64;
Adding route 8
Adding maxpool 9
Adding leaky convolution 10
convBnLeaky: addBatchNorm2d: linx: 10
addBatchNorm2d: lname: module_list.10.BatchNorm2d; shift.count:128;
Adding route 11
Adding leaky convolution 12
convBnLeaky: addBatchNorm2d: linx: 12
addBatchNorm2d: lname: module_list.12.BatchNorm2d; shift.count:64;
Adding leaky convolution 13
convBnLeaky: addBatchNorm2d: linx: 13
addBatchNorm2d: lname: module_list.13.BatchNorm2d; shift.count:64;
Adding route 14
Adding leaky convolution 15
convBnLeaky: addBatchNorm2d: linx: 15
addBatchNorm2d: lname: module_list.15.BatchNorm2d; shift.count:128;
Adding route 16
Adding max pool 17
Adding leaky convolution 18
convBnLeaky: addBatchNorm2d: linx: 18
addBatchNorm2d: lname: module_list.18.BatchNorm2d; shift.count:256;
Adding route 19
Adding leaky convolution 20
convBnLeaky: addBatchNorm2d: linx: 20
addBatchNorm2d: lname: module_list.20.BatchNorm2d; shift.count:128;
Adding leaky convolution 21
convBnLeaky: addBatchNorm2d: linx: 21
addBatchNorm2d: lname: module_list.21.BatchNorm2d; shift.count:128;
Adding route 22
Adding leaky convolution 23
convBnLeaky: addBatchNorm2d: linx: 23
addBatchNorm2d: lname: module_list.23.BatchNorm2d; shift.count:256;
Adding route 24
Adding maxpool 25
Adding leaky convolution 26
convBnLeaky: addBatchNorm2d: linx: 26
addBatchNorm2d: lname: module_list.26.BatchNorm2d; shift.count:512;
Adding leaky convolution 27
convBnLeaky: addBatchNorm2d: linx: 27
addBatchNorm2d: lname: module_list.27.BatchNorm2d; shift.count:256;
Adding leaky convolution 28
convBnLeaky: addBatchNorm2d: linx: 28
addBatchNorm2d: lname: module_list.28.BatchNorm2d; shift.count:512;
Adding linear convolution 29
Adding yolo layer 30
Adding route 31
Adding leaky convolution 32
convBnLeaky: addBatchNorm2d: linx: 32
addBatchNorm2d: lname: module_list.32.BatchNorm2d; shift.count:128;
Adding upsample 33
Adding route 34
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
Adding leaky convolution 35
convBnLeaky: addBatchNorm2d: linx: 35
addBatchNorm2d: lname: module_list.35.BatchNorm2d; shift.count:256;
Adding linear convolution 36
Adding yolo layer 37
Adding route 38
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
[07/24/2021-10:50:21] [E] [TRT] Could not compute dimensions for (Unnamed Layer* 63) [Convolution]_output, because the network is not valid.
[07/24/2021-10:50:21] [E] [TRT] Network validation failed.
main: /workspace/yolov4-triton-tensorrt-r21.05/main.cpp:90: int main(int, char**): Assertion `engine != nullptr' failed.
Aborted (core dumped)

I've added some debug messages to network/yolov4tiny.h to checkpoint the execution. The container got /usr/lib/x86_64-linux-gnu/libnvinfer.so.7.2.2

XiaYuanxiang commented 3 years ago

I got a yolov4tiny.wts file. The engine creation fails with this output:

# ./main --network yolov4tiny
[Info] Creating builder
[Info] Creating model yolov4tiny
convBnLeaky: addBatchNorm2d: linx: 0
addBatchNorm2d: lname: module_list.0.BatchNorm2d; shift.count:32;
convBnLeaky: addBatchNorm2d: linx: 1
addBatchNorm2d: lname: module_list.1.BatchNorm2d; shift.count:64;
convBnLeaky: addBatchNorm2d: linx: 2
addBatchNorm2d: lname: module_list.2.BatchNorm2d; shift.count:64;
convBnLeaky: addBatchNorm2d: linx: 4
addBatchNorm2d: lname: module_list.4.BatchNorm2d; shift.count:32;
convBnLeaky: addBatchNorm2d: linx: 5
addBatchNorm2d: lname: module_list.5.BatchNorm2d; shift.count:32;
Adding route 6
Adding leaky convolution 7
convBnLeaky: addBatchNorm2d: linx: 7
addBatchNorm2d: lname: module_list.7.BatchNorm2d; shift.count:64;
Adding route 8
Adding maxpool 9
Adding leaky convolution 10
convBnLeaky: addBatchNorm2d: linx: 10
addBatchNorm2d: lname: module_list.10.BatchNorm2d; shift.count:128;
Adding route 11
Adding leaky convolution 12
convBnLeaky: addBatchNorm2d: linx: 12
addBatchNorm2d: lname: module_list.12.BatchNorm2d; shift.count:64;
Adding leaky convolution 13
convBnLeaky: addBatchNorm2d: linx: 13
addBatchNorm2d: lname: module_list.13.BatchNorm2d; shift.count:64;
Adding route 14
Adding leaky convolution 15
convBnLeaky: addBatchNorm2d: linx: 15
addBatchNorm2d: lname: module_list.15.BatchNorm2d; shift.count:128;
Adding route 16
Adding max pool 17
Adding leaky convolution 18
convBnLeaky: addBatchNorm2d: linx: 18
addBatchNorm2d: lname: module_list.18.BatchNorm2d; shift.count:256;
Adding route 19
Adding leaky convolution 20
convBnLeaky: addBatchNorm2d: linx: 20
addBatchNorm2d: lname: module_list.20.BatchNorm2d; shift.count:128;
Adding leaky convolution 21
convBnLeaky: addBatchNorm2d: linx: 21
addBatchNorm2d: lname: module_list.21.BatchNorm2d; shift.count:128;
Adding route 22
Adding leaky convolution 23
convBnLeaky: addBatchNorm2d: linx: 23
addBatchNorm2d: lname: module_list.23.BatchNorm2d; shift.count:256;
Adding route 24
Adding maxpool 25
Adding leaky convolution 26
convBnLeaky: addBatchNorm2d: linx: 26
addBatchNorm2d: lname: module_list.26.BatchNorm2d; shift.count:512;
Adding leaky convolution 27
convBnLeaky: addBatchNorm2d: linx: 27
addBatchNorm2d: lname: module_list.27.BatchNorm2d; shift.count:256;
Adding leaky convolution 28
convBnLeaky: addBatchNorm2d: linx: 28
addBatchNorm2d: lname: module_list.28.BatchNorm2d; shift.count:512;
Adding linear convolution 29
Adding yolo layer 30
Adding route 31
Adding leaky convolution 32
convBnLeaky: addBatchNorm2d: linx: 32
addBatchNorm2d: lname: module_list.32.BatchNorm2d; shift.count:128;
Adding upsample 33
Adding route 34
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
Adding leaky convolution 35
convBnLeaky: addBatchNorm2d: linx: 35
addBatchNorm2d: lname: module_list.35.BatchNorm2d; shift.count:256;
Adding linear convolution 36
Adding yolo layer 37
Adding route 38
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: kernel weights has count 0 but 26112 was expected
[07/24/2021-10:50:21] [E] [TRT] (Unnamed Layer* 63) [Convolution]: count of 0 weights in kernel, but kernel dimensions (1,1) with 512 input channels, 51 output channels and 1 groups were specified. Expected Weights count is 512 * 1*1 * 51 / 1 = 26112
[07/24/2021-10:50:21] [E] [TRT] Could not compute dimensions for (Unnamed Layer* 63) [Convolution]_output, because the network is not valid.
[07/24/2021-10:50:21] [E] [TRT] Network validation failed.
main: /workspace/yolov4-triton-tensorrt-r21.05/main.cpp:90: int main(int, char**): Assertion `engine != nullptr' failed.
Aborted (core dumped)

I've added some debug messages to network/yolov4tiny.h to checkpoint the execution. The container got /usr/lib/x86_64-linux-gnu/libnvinfer.so.7.2.2

Do you solve this problem？I meet the same problem, any help will appreciate

GowthamKudupudi commented 3 years ago

I fixed many of the [E] [TRT]s by replacing the model with module_list in network/yolov4tiny.h which I found in yolov4tiny.wts. and then I bypassed few of the assertions to successfully create the .engine file but the .engine is unable to infer properly now.

XiaYuanxiang commented 3 years ago

replace model with module_list, bn with BatchNorm2d, conv with Conv2d can fix this problem

GowthamKudupudi commented 3 years ago

@XiaYuanxiang, did your .engine file infering properly now? Please post me your du -h yolov4tiny.wts and du -h yolov4tiny.engine.

XiaYuanxiang commented 3 years ago

yes, it works properly now, but I'm using yolov4. Just try the method I said before