How can I use this repo for video inferencing?

[Bao150297]

I want to use this repo to detect object from video stream or from CSI camera. But I'm not sure whether I can and what to do to perform that idea. Can anyone share with me?

[maazqureshi446]

I have also same question as DoriHP. Any help will be appreciated.

[Bao150297]

I trained a ssd-mobilenet-v1 model for only 1 class. It worked well while inside jetson-inference project, so I thought I can try using it with this repo. I tried but the result was bad, the model created many bounding box around the object

[maazqureshi446]

Dear DoriHp, Could you please share the reference to run customize model with jetson-inference. Thanks in advance

[Bao150297]

I'm in progress to make it into a completed project but I have some suggestions for you: you only need to retrain one of the pretrained models that project give you, after that, use this repo to convert your custom model into uff format. Finally, replace the original model in that project by your own, and enjoy your work. If you have any question, feel free to ask me, and I will reply ( If I can :v )

[Bao150297]

Still wait for the answer from @AastaNV . I tried but still get failure.

[maazqureshi446]

Dear DoriHp, I need a little more detail about conversion of frozen_inference_graph.pb (customize) to .uff format. I used this file for conversion but failed: /usr/lib/python3.5/dist-packages/uff/bin/convert_to_uff.py

Thanks in advance.

[maazqureshi446]

I retrained SSD_MOBILENET_V2_COCO model and now I want to convert my customize model .pb file to .uff and then use detectnet-camera.py ( Jetson-Inference ) to load my customize model (.uff).

[Bao150297]

Have you done your work? If not yet, I strongly recommend using this config for ssd_mobilenet_v2_coco of this repo for converting your model to uff format. And the another point, for reproducing the model that is compatible with this config, you should use train.py instead of model_main.py during training process.

[revrr]

I managed to convert a mobilenet_v2_coco .pb model into .uff and it worked on the jetson-inference. but when I try to use this repo to convert my own mobilenet_v2 model it outputs the following:

$ python3 main.py 000012.jpg 
[TensorRT] ERROR: Could not register plugin creator:  FlattenConcat_TRT in namespace: 
WARNING: To create TensorRT plugin nodes, please use the `create_plugin_node` function instead.
NOTE: UFF has been tested with TensorFlow 1.14.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.5
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
input: "Cast"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]
=========================================

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_4_3x3_s2_256/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_4_1x1_128/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_3_3x3_s2_256/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_3_1x1_128/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_2_3x3_s2_512/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_2_1x1_256/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/Conv_1/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_16/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_16/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_16/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_15/add as custom op: AddV2
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_14/add as custom op: AddV2
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_13/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_13/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_13/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_12/add as custom op: AddV2
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_11/add as custom op: AddV2
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_10/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_10/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_10/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_9/add as custom op: AddV2
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_8/add as custom op: AddV2
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_7/add as custom op: AddV2
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_6/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_6/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_6/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_5/add as custom op: AddV2
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_4/add as custom op: AddV2
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_3/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_3/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_3/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: AddV2 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_2/add as custom op: AddV2
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_1/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_1/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_1/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: Cast yet.
Converting Cast as custom op: Cast
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_2/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_2/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_2/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_4/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_4/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_4/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_5/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_5/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_5/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_7/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_7/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_7/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_8/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_8/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_8/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_9/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_9/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_9/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_11/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_11/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_11/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_12/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_12/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_12/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_14/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_14/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_14/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_15/project/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_15/depthwise/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: FusedBatchNormV3 yet.
Converting FeatureExtractor/MobilenetV2/expanded_conv_15/expand/BatchNorm/FusedBatchNormV3 as custom op: FusedBatchNormV3
Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting GridAnchor as custom op: GridAnchor_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
DEBUG [/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:96] Marking ['NMS'] as outputs
No. nodes: 644
UFF Output written to tmp.uff
[TensorRT] ERROR: UffParser: Validator error: FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_4_3x3_s2_256/BatchNorm/FusedBatchNormV3: Unsupported operation _FusedBatchNormV3
[TensorRT] ERROR: Network must have at least one output
Traceback (most recent call last):
  File "main.py", line 43, in <module>
    buf = engine.serialize()
AttributeError: 'NoneType' object has no attribute 'serialize'

and the .uff doesn't load on the jetson-inference.

I'm training with legacy train.py and still not working

[Bao150297]

I recommend that you should roll back for the earlier commit of TensorFlow Object Detection API since some layers in the same network have been changed in newer version. I did it and it work very well (at least with the model I want to use - SSD MobileNet v1)