Launch object detection with custom .xml model

[charlotteheggem]

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes OS Platform and Distribution: Ubuntu 18.04 ROS2 distribution: Eloquent TensorFlow installed from (source or binary): binary TensorFlow version: 1.14.0 Python version: 3.6.9 ros2_openvino_toolkit branch: Eloquent opencv/dldt branch: 2020

I have edited the pipeline_object_topic.yaml file to take in a customized model made with tensorflow 1.14. The model is converted to .xml file by using the mo_tf.py script (https://github.com/opencv/dldt/blob/2020/model-optimizer/mo_tf.py) by:

sudo python3 mo_tf.py --input_model=/home/charlotte/MyDataset/test/model/model.pb --tensorflow_use_custom_operations_config /home/charlotte/opencv/dldt/model-optimizer/extensions/front/tf/ssd_support_api_v1.14.json --tensorflow_object_detection_api_pipeline_config /home/charlotte/MyDataset/test/model/pipeline.config --output_dir /home/charlotte/MyDataset

This works well, but I get the following error when launching ros2 launch dynamic_vino_sample pipeline_object_topic.launch.py: [pipeline_with_params-1] [ ERROR ] Error reading network: cannot parse future versions: 10

I tried changing the IR_version to generate a xml file with IR_version 5, by changing the returning version in this file: https://github.com/opencv/dldt/blob/2020/model-optimizer/mo/pipeline/common.py, which results in a the following header in my .xml file:

When I try to launch the object detection node with this model, I get the following error: charlotte@charlotte-MacBookAir:~/opencv/dldt/model-optimizer$ ros2 launch dynamic_vino_sample pipeline_object_topic.launch.py [INFO] [launch]: All log files can be found below /home/charlotte/.ros/log/2020-03-27-16-58-28-573633-charlotte-MacBookAir-30266 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [pipeline_with_params-1]: process started with pid [30280] [pipeline_with_params-1] InferenceEngine: 0x7f34e2b3b060 [pipeline_with_params-1] [ INFO ] Config File Path =/home/charlotte/ros2_overlay_ws/install/dynamic_vino_sample/share/dynamic_vino_sample/param/pipeline_object_topic.yaml [pipeline_with_params-1] [ INFO ] Pipeline size: 1 [pipeline_with_params-1] [ INFO ] Inferences size: 1 [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ INFO ] Inference Params:name=ObjectDetection [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ INFO ] Pipeline Params:name=object [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ WARNING ] invalid node; this may result from using a map iterator as a sequence iterator, or vice-versa [pipeline_with_params-1] [ INFO ] --------parameters DUMP--------------------- [pipeline_with_params-1] [ INFO ] Pipeline: object [pipeline_with_params-1] [ INFO ] Inputs: RealSenseCameraTopic, [pipeline_with_params-1] [ INFO ] Outputs: ImageWindow, RosTopic, RViz, [pipeline_with_params-1] [ INFO ] Inferences: [pipeline_with_params-1] [ INFO ] Name: ObjectDetection [pipeline_with_params-1] [ INFO ] Model: /home/charlotte/MyDataset/test/model/model.xml [pipeline_with_params-1] [ INFO ] Engine: CPU [pipeline_with_params-1] [ INFO ] Label: to/be/set/xxx.labels [pipeline_with_params-1] [ INFO ] Batch: 1 [pipeline_with_params-1] [ INFO ] Confidence_threshold: 0.5 [pipeline_with_params-1] [ INFO ] Enable_roi_constraint: 1 [pipeline_with_params-1] [ INFO ] Connections: [pipeline_with_params-1] [ INFO ] ObjectDetection->ImageWindow [pipeline_with_params-1] [ INFO ] ObjectDetection->RosTopic [pipeline_with_params-1] [ INFO ] ObjectDetection->RViz [pipeline_with_params-1] [ INFO ] RealSenseCameraTopic->ObjectDetection [pipeline_with_params-1] [ INFO ] Common: [pipeline_with_params-1] [ INFO ] camera_topic: [pipeline_with_params-1] [ INFO ] custom_cpu_library: [pipeline_with_params-1] [ INFO ] custom_cldnn_library: [pipeline_with_params-1] [ INFO ] enable_performance_count: 0 [pipeline_with_params-1] [ INFO ] Parsing InputDvice: RealSenseCameraTopic [pipeline_with_params-1] [ INFO ] ... Adding one Input device: RealSenseCameraTopic [pipeline_with_params-1] [ INFO ] Adding Input Device into Pipeline: RealSenseCameraTopic [pipeline_with_params-1] [ INFO ] Adding connection into pipeline:[<-->RealSenseCameraTopic] [pipeline_with_params-1] [ INFO ] Parsing Output: ImageWindow [pipeline_with_params-1] [ INFO ] ... Adding one Output: ImageWindow [pipeline_with_params-1] [ INFO ] Parsing Output: RosTopic [pipeline_with_params-1] [ INFO ] ... Adding one Output: RosTopic [pipeline_with_params-1] [ INFO ] Parsing Output: RViz [pipeline_with_params-1] [ INFO ] ... Adding one Output: RViz [pipeline_with_params-1] [ INFO ] Parsing Inference: ObjectDetection [pipeline_with_params-1] [ INFO ] Loading network files [pipeline_with_params-1] [ ERROR ] Error reading network: Error of validate layer: BoxPredictor_0/BoxEncodingPredictor/Conv2D with type: Convolution. No such parameter name 'output' for layer BoxPredictor_0/BoxEncodingPredictor/Conv2D [ERROR] [pipeline_with_params-1]: process has died [pid 30280, exit code 254, cmd '/home/charlotte/ros2_overlay_ws/install/dynamic_vino_sample/lib/dynamic_vino_sample/pipeline_with_params -config /home/charlotte/ros2_overlay_ws/install/dynamic_vino_sample/share/dynamic_vino_sample/param/pipeline_object_topic.yaml --ros-args -r /openvino_toolkit/object/detected_objects:=/ros2_openvino_toolkit/detected_objects -r /openvino_toolkit/object/images:=/ros2_openvino_toolkit/image_rviz']. I have no problems running ros2 launch dynamic_vino_sample pipeline_object_topic.launch.py with one of the example models, such as the mobilenet-ssd from here: https://github.com/intel/ros2_openvino_toolkit/blob/master/doc/OBJECT_DETECTION.md. I`ve tried removing the output from the layer concerned and the edge for the output from my model.xml file, but the error remains. Is the problem that ros2_openvino_toolkit is not compatible with the 2020 openvino distribution? Attached is my model.xml file: [model.txt](https://github.com/intel/ros2_openvino_toolkit/files/4397017/model.txt)

[LewisLiuPub]

hi, @charlotteheggem ROS2 OpenVINO supports SSD-like Detectin Models by default, so if the model you customized doesn't follow the inputs/outputs as SSD/yolo, you need to update source code to align with your customization.