Mobilenetv3 - bad results on Jetson Xavier

[danwanban]

Platform Jetpack 5, Jetson Xavier, Cuda 11.4, TRT 8.4.1 Latest branch Mobilenetv3 - Getting strange results

Hi,

Getting strange results: Step 1: D/n https://github.com/chufei1995/mobilenetv3.pytorch, used the pretrained weigths and created wts using:

from mobilenetv3 import mobilenetv3_large, mobilenetv3_small
import torch
import struct
net_small = mobilenetv3_small()
net_small.load_state_dict(torch.load('pretrained/mobilenetv3-small-c7eb32fe.pth'))

f = open('mobilenetv3_small.wts', 'w')
f.write("{}\n".format(len(net_small.state_dict().keys())))
for k,v in net_small.state_dict().items():
  vr = v.reshape(-1).cpu().numpy()
  f.write("{} {}".format(k, len(vr)))
  for vv in vr:
    f.write(" ")
    f.write(struct.pack(">f", float(vv)).hex())
  f.write("\n")

the mobilenetv3_small.wts (size,md5) is (28167639,081ce283457e184d0bf6ab1a50aca2b2)

Step 2: Created the engine - ran ./mobilenetv3 -s small:

small
create engine small
[12/19/2022-11:04:30] [W] [TRT] The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
Loading weights: ../mbv3_small.wts
len 16
len 16
len 16
len 72
len 72
len 24
len 88
len 88
len 24
len 96
len 96
len 40
len 240
len 240
len 40
len 240
len 240
len 40
len 120
len 120
len 48
len 144
len 144
len 48
len 288
len 288
len 96
len 576
len 576
len 96
len 576
len 576
len 96
len 576
len 1280
len 1000
set name out
build out

engine size,md5: 13409326, 4b196d90f1a2e9d8b69d7285cd6036ff (I am aware these size/md5 might be effected by randomness and platform)

Step 3: tried to run on an image of a cat:

...
cv::Mat img_raw = cv::imread(argv[3]);
cv::Mat img;
resize(img_raw, img, cv::Size(INPUT_W, INPUT_H), cv::INTER_LINEAR);
int b=0;
int i = 0;
for (int row = 0; row < INPUT_H; ++row) {
  uchar* uc_pixel = img.data + row * img.step;
  for (int col = 0; col < INPUT_W; ++col) {
    data[b * 3 * INPUT_H * INPUT_W + i] = ((float)(uc_pixel[2]-123.675) /255.0);
    data[b * 3 * INPUT_H * INPUT_W + i + INPUT_H * INPUT_W] = ((float)(uc_pixel[1]- 116.28)/255.0);
    data[b * 3 * INPUT_H * INPUT_W + i + 2 * INPUT_H * INPUT_W] = ((float)(uc_pixel[0]-103.53)/255.0);
    uc_pixel += 3;
    ++i;
  }
}       

// continue as the sample code

I'm getting strange results (Prob >1 , incorrect): Output: -1.4593, 3.17417, 0.276502, 0.436363, 2.20322, 4.69737, 6.74635, 0.892753, 2.66647, -1.66991, -1.15208, -3.03486, 0.169612, 0.315448, -2.13814, -0.919521, -2.16002, -1.30154, -1.32885, -4.86003, -2.53131, -1.30849, -2.0221, -0.568821, -2.98014, -4.80999, -2.57325, -0.511288, -5.15728, -0.332619, -4.88908, -3.43442, -3.91384, -2.80743, -2.72072, -4.29223, -2.15746, -2.17751, 0.432898, 0.474316, 0.661687, 1.0473, 1.29892, -1.54822, -0.462609, -3.1105, 1.61716, -0.815477, -3.22607, -6.46574, -3.66524, -1.11003, 0.0253144, -1.15286, 0.885665, -1.34477, -2.9329, -3.09455, -1.25512, -1.99589, -0.889628, -1.2278, -2.36516, -1.91292, 0.391547, -0.490322, 1.13408, 2.9246, -0.668016, 0.62884, 2.76731, 0.75406, -0.182454, 1.09703, 1.01042, 2.61895, -0.64699 ....

Same if I just normalize (uc_pixel[2] / 255.0)

Step 4: Tried to run on the "only ones vector", When I am running on the "1" vector I also get non-prob results:

...
for (int i = 0; i < 3 * INPUT_H * INPUT_W; i++)
  data[i] = 1.0;
...

Output: 8.51321, 8.34527, 7.92485, 7.87945, 7.3711, 8.87762, 7.3669, 7.46516, 7.35648, 8.39938, 9.64395, 8.90672, 8.06779, 8.96273, 8.16083, 7.74594, 8.77602, 8.01993, 8.58591, 8.65454, 8.01972, 11.2975, 9.98022, 9.54374, 8.0043, 7.6174, 8.59277, 7.42256, 7.67885, 8.03237, 7.23091, 7.5367, 7.1943, 8.91155, 9.22677, 7.35447, 8.11877, 7.34899, 9.62806, 8.78016, 8.392, 8.89872, 10.3375, 9.0303, 8.33717, 8.3553, 8.38832, 8.74276, 7.51678, 7.89853, 8.91163, 9.17735, 8.65279, 8.71997, 8.64744, 8.46851, 8.67575, 7.9884, 8.74841, 10.0022, 8.97073, 8.92307, 9.0762, 9.66539, 9.36507, 8.37905, 9.27205, 8.55999, 9.6262, 8.66072, 8.52823, 8.4032, 8.78367, 10.1793, 8

Same results in large as well.

Any idea ?

Thanks for the help !

[wang-xinyu]

Don't we need to add a softmax layer at the end of the network?

Can you double check the pytorch code?

[danwanban]

Added the softmax, it seems working ok with the normalization. Thanks !