Add program to ck-tensorrt for classification based on TensorRT

DVEfremov commented 7 years ago

Preconditions: git cmake
Add package based on https://github.com/dusty-nv/jetson-inference
add install script for jetson-inference git clone https://github.com/dusty-nv/jetson-inference mkdir build cd build cmake ../ ** make
Add program imagenet-console located after build at jetson-inference/build/aarch64/bin with one action? example cmd: $ ./imagenet-console orange_0.jpg output_0.jpg And ou put like:

imagenet-console
  args (3):  0 [./imagenet-console]  1 [orange_0.jpg]  2 [output_0.jpg]  

[GIE]  attempting to open cache file bvlc_googlenet.caffemodel.tensorcache
[GIE]  cache file not found, profiling network model
[GIE]  platform has FP16 support.
[GIE]  loading googlenet.prototxt bvlc_googlenet.caffemodel
[GIE]  retrieved output tensor 'prob'
[GIE]  configuring CUDA engine
[GIE]  building CUDA engine
[GIE]  completed building CUDA engine
[GIE]  network profiling complete, writing cache to bvlc_googlenet.caffemodel.tensorcache
[GIE]  completed writing cache to bvlc_googlenet.caffemodel.tensorcache
[GIE]  bvlc_googlenet.caffemodel loaded
[GIE]  CUDA engine context initialized with 2 bindings
[GIE]  bvlc_googlenet.caffemodel input  binding index:  0
[GIE]  bvlc_googlenet.caffemodel input  dims (c=3 h=224 w=224) size=602112
[cuda]  cudaAllocMapped 602112 bytes, CPU 0x100be0000 GPU 0x100be0000
[GIE]  bvlc_googlenet.caffemodel output 0 prob  binding index:  1
[GIE]  bvlc_googlenet.caffemodel output 0 prob  dims (c=1000 h=1 w=1) size=4000
[cuda]  cudaAllocMapped 4000 bytes, CPU 0x100ce0000 GPU 0x100ce0000
bvlc_googlenet.caffemodel initialized.
[GIE]  googlenet loaded
imageNet -- loaded 1000 class info entries
googlenet initialized.
loaded image  orange_0.jpg  (1920 x 1920)  58982400 bytes
[cuda]  cudaAllocMapped 58982400 bytes, CPU 0x100de0000 GPU 0x100de0000
class 0950 - 0.978516  (orange)
class 0951 - 0.021271  (lemon)
imagenet-console:  'orange_0.jpg' -> 97.85156% class #950 (orange)
loaded image  fontmapA.png  (256 x 512)  2097152 bytes
[cuda]  cudaAllocMapped 2097152 bytes, CPU 0x104620000 GPU 0x104620000
[cuda]  cudaAllocMapped 8192 bytes, CPU 0x104820000 GPU 0x104820000
imagenet-console:  attempting to save output image to 'output_0.jpg'
imagenet-console:  completed saving 'output_0.jpg'

shutting down...

Known issues to be done in scope of this task:

fix issue if ck-caffe does not exits with

ck install package:lib-jetson-inference-dusty-nv-master-cuda
..
[100%] Built target v4l2-display
CK error: [package] Software environment UOA is not defined in this package (soft_uoa)!

remove hardcoded CK_ENV_COMPILER_CUDA
pass image and network model as params to own imagenet-console program using ck data sets (model.caffe)
Support two versions: GIE (1.0) и TensorRT (2.0)

DVEfremov commented 7 years ago

Looks like it uses model located at the same dir as imagenet-console:

daniil@tegra-ubuntu:~/tensorrt/jetson-inference/build/aarch64/bin$ ls -l
total 322280
-rw-r--r-- 1 daniil dvdt      3629 Dec 14 03:26 alexnet.prototxt
-rw-r--r-- 1 daniil dvdt     79281 Dec 14 03:26 banana_0.jpg
-rw-r--r-- 1 daniil dvdt 243862414 Dec 14 03:32 bvlc_alexnet.caffemodel
-rw-r--r-- 1 daniil dvdt  53533754 Dec 14 03:32 bvlc_googlenet.caffemodel
-rw-r--r-- 1 daniil dvdt  14033968 Dec 14 04:44 bvlc_googlenet.caffemodel.tensorcache
-rwxr-xr-x 1 daniil dvdt    540880 Dec 14 04:42 detectnet-camera
-rwxr-xr-x 1 daniil dvdt    534944 Dec 14 04:42 detectnet-console
drwxr-xr-x 2 daniil dvdt      4096 Dec 14 03:36 facenet-120
drwxr-xr-x 2 daniil dvdt      4096 Dec 14 03:36 FCN-Alexnet-Cityscapes-HD
drwxr-xr-x 2 daniil dvdt      4096 Dec 14 03:36 FCN-Alexnet-SYNTHIA-Summer-HD
-rw-r--r-- 1 daniil dvdt     13310 Dec 14 03:26 fontmapA.png
-rw-r--r-- 1 daniil dvdt     13902 Dec 14 03:26 fontmapB.png
-rw-r--r-- 1 daniil dvdt     35861 Dec 14 03:26 googlenet.prototxt
-rw-r--r-- 1 daniil dvdt     14318 Dec 14 03:26 granny_smith_0.jpg
-rw-r--r-- 1 daniil dvdt    547142 Dec 14 03:26 granny_smith_1.jpg
-rwxr-xr-x 1 daniil dvdt    535432 Dec 14 04:42 gst-camera
-rw-r--r-- 1 daniil dvdt     31675 Dec 14 03:26 ilsvrc12_synset_words.txt
-rwxr-xr-x 1 daniil dvdt    536888 Dec 14 04:42 imagenet-camera
-rwxr-xr-x 1 daniil dvdt    530872 Dec 14 04:42 imagenet-console
drwxr-xr-x 2 daniil dvdt      4096 Dec 14 03:36 multiped-500
-rw-r--r-- 1 daniil dvdt    997075 Dec 14 03:26 orange_0.jpg
-rw-r--r-- 1 daniil dvdt    187389 Dec 14 03:26 orange_1.jpg
-rw-r--r-- 1 daniil dvdt    233213 Dec 14 04:44 output_0.jpg
drwxr-xr-x 2 daniil dvdt      4096 Dec 14 03:36 ped-100
-rw-r--r-- 1 daniil dvdt    851274 Dec 14 03:26 peds-001.jpg
-rw-r--r-- 1 daniil dvdt    791354 Dec 14 03:26 peds-002.jpg
-rw-r--r-- 1 daniil dvdt    224055 Dec 14 03:26 peds-003.jpg
-rw-r--r-- 1 daniil dvdt   2152317 Dec 14 03:26 peds-004.png
-rw-r--r-- 1 daniil dvdt   2237955 Dec 14 03:26 peds-005.png
-rw-r--r-- 1 daniil dvdt   1908379 Dec 14 03:26 peds-006.png
-rw-r--r-- 1 daniil dvdt   1954575 Dec 14 03:26 peds-007.png
-rw-r--r-- 1 daniil dvdt   2250093 Dec 14 03:26 peds-008.png
-rw-r--r-- 1 daniil dvdt    726779 Dec 14 03:26 red_apple_0.jpg
-rwxr-xr-x 1 daniil dvdt       121 Dec 14 03:26 segnet-batch.sh
-rwxr-xr-x 1 daniil dvdt    530640 Dec 14 04:42 segnet-console
-rwxr-xr-x 1 daniil dvdt     19920 Dec 14 04:42 v4l2-console
-rwxr-xr-x 1 daniil dvdt     14704 Dec 14 04:42 v4l2-display

psyhtest commented 7 years ago

I've modified the package to skip downloading the models, as we should reuse the ones available as CK packages. Then I modified the program to set up the ImageNet object with the paths specified by CK.

On the first launch, a TensorRT model is created:

      [GIE]  attempting to open cache file /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel.tensorcache
      [GIE]  cache file not found, profiling network model
      [GIE]  platform has FP16 support.
      [GIE]  loading /home/anton/CK_REPOS/ck-tensorrt/program/imagenet-console/tmp/tmp-5AGYLd.prototxt /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/b
vlc_alexnet.caffemodel
      [GIE]  retrieved output tensor 'prob'
      [GIE]  configuring CUDA engine
      [GIE]  building CUDA engine
      [GIE]  completed building CUDA engine
      [GIE]  network profiling complete, writing cache to /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel.tensorcache
      [GIE]  completed writing cache to /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel.tensorcache
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel loaded
      [GIE]  CUDA engine context initialized with 2 bindings
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel input  binding index:  0
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel input  dims (c=3 h=227 w=227) size=618348
      [cuda]  cudaAllocMapped 618348 bytes, CPU 0x100be0000 GPU 0x100be0000
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel output 0 prob  binding index:  1
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel output 0 prob  dims (c=1000 h=1 w=1) size=4000
      [cuda]  cudaAllocMapped 4000 bytes, CPU 0x100ce0000 GPU 0x100ce0000
      /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel initialized.
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel loaded
      imageNet -- loaded 1000 class info entries
      /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel initialized.
      loaded image  /home/anton/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-dnn-cat/cat.jpg  (480 x 360)  2764800 bytes
      [cuda]  cudaAllocMapped 2764800 bytes, CPU 0x100de0000 GPU 0x100de0000
      class 0151 - 0.028091  (Chihuahua)
      class 0263 - 0.026367  (Pembroke, Pembroke Welsh corgi)
      class 0274 - 0.033081  (dhole, Cuon alpinus)
      class 0277 - 0.201172  (red fox, Vulpes vulpes)
      class 0278 - 0.044891  (kit fox, Vulpes macrotis)
      class 0280 - 0.010330  (grey fox, gray fox, Urocyon cinereoargenteus)
      class 0281 - 0.072876  (tabby, tabby cat)
      class 0282 - 0.100342  (tiger cat)
      class 0285 - 0.207520  (Egyptian cat)
      class 0287 - 0.015884  (lynx, catamount)
      class 0330 - 0.048157  (wood rabbit, cottontail, cottontail rabbit)
      class 0331 - 0.072266  (hare)
      class 0356 - 0.015388  (weasel)
      class 0904 - 0.024399  (window screen)
      imagenet-console:  '/home/anton/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-dnn-cat/cat.jpg' -> 20.75195% class #285 (Egyptian cat)

On subsequent launches, the TensorRT model is reused:

      [GIE]  attempting to open cache file /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel.tensorcache
      [GIE]  loading network profile from cache... /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel.tensorcache
      [GIE]  platform has FP16 support.
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel loaded
      [GIE]  CUDA engine context initialized with 2 bindings
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel input  binding index:  0
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel input  dims (c=3 h=227 w=227) size=618348
      [cuda]  cudaAllocMapped 618348 bytes, CPU 0x100be0000 GPU 0x100be0000
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel output 0 prob  binding index:  1
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel output 0 prob  dims (c=1000 h=1 w=1) size=4000
      [cuda]  cudaAllocMapped 4000 bytes, CPU 0x100ce0000 GPU 0x100ce0000
      /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel initialized.
      [GIE]  /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel loaded
      imageNet -- loaded 1000 class info entries
      /home/anton/CK_TOOLS/caffemodel-bvlc-alexnet/bvlc_alexnet.caffemodel initialized.
      loaded image  /home/anton/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-dnn-cat/cat.jpg  (480 x 360)  2764800 bytes
      [cuda]  cudaAllocMapped 2764800 bytes, CPU 0x100de0000 GPU 0x100de0000
      class 0151 - 0.028091  (Chihuahua)
      class 0263 - 0.026367  (Pembroke, Pembroke Welsh corgi)
      class 0274 - 0.033081  (dhole, Cuon alpinus)
      class 0277 - 0.201172  (red fox, Vulpes vulpes)
      class 0278 - 0.044891  (kit fox, Vulpes macrotis)
      class 0280 - 0.010330  (grey fox, gray fox, Urocyon cinereoargenteus)
      class 0281 - 0.072876  (tabby, tabby cat)
      class 0282 - 0.100342  (tiger cat)
      class 0285 - 0.207520  (Egyptian cat)
      class 0287 - 0.015884  (lynx, catamount)
      class 0330 - 0.048157  (wood rabbit, cottontail, cottontail rabbit)
      class 0331 - 0.072266  (hare)
      class 0356 - 0.015388  (weasel)
      class 0904 - 0.024399  (window screen)
      imagenet-console:  '/home/anton/CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-dnn-cat/cat.jpg' -> 20.75195% class #285 (Egyptian cat)

This is with AlexNet and $CK_REPOS/ctuning-datasets-min/dataset/image-jpeg-dnn-cat/cat.jpg (do ck pull repo:ctuning-datasets-min to get this image).

psyhtest commented 7 years ago

The new tensorrt-test program is now complete. It supports classifying individial images, as well as scoring the top1 and top5 accuracy on the ImageNet validation set (50,000 images).

ctuning / ck-tensorrt

Add program to ck-tensorrt for classification based on TensorRT #1