nnstreamer / nnstreamer-example

Example applications of nnstreamer. Note that we have to enable the 'apptest" CI module in the near future.
GNU Lesser General Public License v2.1
79 stars 71 forks source link

[WIP][bashscript/android] Support tflite segmentation model #69

Closed niklasjang closed 5 years ago

niklasjang commented 5 years ago

It is a sub-issue of #62 Support tensorflow-lite basic model with bash_script .

As mentioned in style transfer reallocate issue, there is a realloc problem with style transfer model. So I am going to try to support segmentation example.

I am considering this segmentation model which is supported by tensorflow.

//supported labels
LABEL_NAMES = np.asarray([
    'background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus',
    'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike',
    'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tv'
])

Brief pipeline

gst-launch-1.0 -v \
      filesrc location=cat.jpg ! jpegdec ! videoconvert ! videoscale ! imagefreeze !\
      video/x-raw,format=RGB,width=257,height=257,framerate=10/1 ! tee name=t \
      t. tensor_converter !\
      tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 !\
      tensor_filter framework=tensorflow-lite model=deeplabv3_257_mv_gpu.tflite !\
      tensor_decoder mode=segmentation ! compositor name=mix sink_0::zorder=2 sink_1::zorder=1 !\
      videoconvert ! ximagesink
      t. ! queue leaky=2 ! mix.

At first, I will make decoder(output array -> image type) for this model and then construct bash script example. If it is done, I will expand it to android application example.

taos-ci commented 5 years ago

:octocat: cibot: Thank you for posting issue #69. The person in charge will reply soon.

niklasjang commented 5 years ago

This feature is merged at (https://github.com/nnsuite/nnstreamer/pull/1801).

I'm closing this.