google / automl

Google Brain AutoML
Apache License 2.0
6.18k stars 1.45k forks source link

Error reading original efficientdet-d3_frozen.pb on openCV`s readNetFromTensorflow #1206

Closed stuurgurs closed 6 months ago

stuurgurs commented 6 months ago

Error reading original efficientdet-d3_frozen.pb on openCV`s readNetFromTensorflow.

During the process, I successfully converted the original coco checkpoint of the efficientdet-d3 dataset into a frozen tensorflow model:

python model_inspect.py --runmode=saved_model --model_name=efficientdet-d3 --ckpt_path=efficientdet-d3 --saved_model_dir=export_orig Python 3.9.11 tensorflow>=2.10.0

Successfully tested the file on a test image - detection works python model_inspect.py --runmode=saved_model_infer --model_name=efficientdet-d3 --saved_model_dir=export_orig\efficientdet-d3_frozen.pb --min_score_thresh=0.1 --input_image=test2.jpg --output_image_dir=output

config.yaml

input_rand_hflip: True jitter_max: 1 jitter_min: 1 num_classes: 2 max_level: 7 num_scales: 2 anchor_scale: 4 skip_crowd_during_training: False label_map: {0: background, 1: ball}

0

Then strange things happen: We take the frozen model and try to use it in the project Opencv 4.6.0 CUDA 11.4 Ubuntu 20.04 Python 3.8.10

Let's run the code: let net = await cv2.readNetFromTensorflow(modelPbPathOrig);

We get an error: [ERROR:0@0.587] global /home/user/openc v4/opencv-4.6.0/modules/dnn/src/tensorflow/tf_importer.cpp (3158) parseNode DNN/TF: Can't parse layer for node='strided_slice_21' of type='StridedSlice'. Exception: OpenCV Error: (Input layer not found: Shape_8) in connect, in file /home/user/opencv4/opencv-4.6.0/modules/dnn/src/tensorflow/tf_importer.cpp, line 2799, status -2

What is the problem I can’t understand?

I tried to create a frozen model file with different hyperparameters, with a different number of classes and other settings - get an error.

I also tried to create and use a graph file .pbtxt with .pb - get an error.

I tried to use other opencv_face_detector_uint8.pb model files - it loads without errors!

stuurgurs commented 6 months ago

The solution was to perform tensorFlow graph transform with tensorflow/tools/graph_transforms/transform_graph

bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=/home/user/panel/server/cascades/savedmodel/efficientdet-d0_frozen.pb --out_graph=/home/user/panel/server/cascades/savedmodel/efficientdet-d0_frozen2.pb --inputs="input" --outputs="class_net/class-predict/BiasAdd,class_net/class-predict_1/BiasAdd,class_net/class-predict_2/BiasAdd,class_net/class-predict_3/BiasAdd,class_net/class-predict_4/BiasAdd,box_net/box-predict/BiasAdd,box_net/box-predict_1/BiasAdd,box_net/box-predict_2/BiasAdd,box_net/box-predict_3/BiasAdd,box_net/box-predict_4/BiasAdd" --transforms="fold_constants"

also you should take all outputs. Yu can see your graph outputs by

bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=/home/user/panel/server/cascades/savedmodel/efficientdet-d0_frozen.pb

Found 10 possible outputs: (name=class_net/class-predict/BiasAdd, op=BiasAdd) (n                                   ame=class_net/class-predict_1/BiasAdd, op=BiasAdd) (name=class_net/class-predict                                   _2/BiasAdd, op=BiasAdd) (name=class_net/class-predict_3/BiasAdd, op=BiasAdd) (na                                   me=class_net/class-predict_4/BiasAdd, op=BiasAdd) (name=box_net/box-predict/Bias                                   Add, op=BiasAdd) (name=box_net/box-predict_1/BiasAdd, op=BiasAdd) (name=box_net/                                   box-predict_2/BiasAdd, op=BiasAdd) (name=box_net/box-predict_3/BiasAdd, op=BiasA                                   dd) (name=box_net/box-predict_4/BiasAdd, op=BiasAdd)
to build this tool you can run on ububtu cmd next command:

bazel build tensorflow/tools/graph_transforms:summarize_graph

And also change OpenCV version to 4.7.0

stuurgurs commented 6 months ago

solved