search

onnx / tensorflow-onnx

Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX
Apache License 2.0
2.31k stars 430 forks source link

Problem converting TensorFlow2 model to Onnx #847

Closed jackyvr closed 4 years ago

jackyvr commented 4 years ago

I tried converting a tf2 model (yolov3) to onnx, but failed. Below is the console log? Any ideas? Thanks!!

Script: python -m tf2onnx.convert --saved-model /yolov3/1 --output tests/models/fc-layers/model.onnx --verbose

Console: 2020-03-16 13:34:31.653818: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll Traceback (most recent call last): File "D:\software\conda_envs\yolov3-tf2-gpu\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "D:\software\conda_envs\yolov3-tf2-gpu\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\convert.py", line 169, in main() File "D:\work\cv\others\tensorflow-onnx\tf2onnx\convert.py", line 130, in main args.saved_model, args.inputs, args.outputs, args.signature_def) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tf_loader.py", line 217, in from_saved_model _from_saved_model_v2(model_path, input_names, output_names, signatures) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tf_loader.py", line 192, in _from_saved_model_v2 imported = tf.saved_model.load(model_path) # pylint: disable=no-value-for-parameter File "D:\software\conda_envs\yolov3-tf2-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 517, in load return load_internal(export_dir, tags) File "D:\software\conda_envs\yolov3-tf2-gpu\lib\site-packages\tensorflow_core\python\saved_model\load.py", line 526, in load_internal saved_model_proto = loader_impl.parse_saved_model(export_dir) File "D:\software\conda_envs\yolov3-tf2-gpu\lib\site-packages\tensorflow_core\python\saved_model\loader_impl.py", line 83, in parse_saved_model constants.SAVED_MODEL_FILENAME_PB)) OSError: SavedModel file does not exist at: /yolov3/1/saved_model.pb/{saved_model.pbtxt|saved_model.pb}

(D:\software\conda_envs\yolov3-tf2-gpu) D:\work\cv\others\tensorflow-onnx>python -m tf2onnx.convert --saved-model /yolov3/1 --output tests/models/fc-layers/model.onnx --verbose 2020-03-16 13:34:45.532513: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll 2020-03-16 13:34:47.383460: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll 2020-03-16 13:34:47.482412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:34:47.486208: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:34:47.488384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:34:47.490667: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 2020-03-16 13:34:47.494297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:34:47.496833: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:34:47.499071: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:34:48.088700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-16 13:34:48.091093: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-03-16 13:34:48.092289: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2020-03-16 13:34:48.093799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6351 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1) 2020-03-16 13:34:59.552310: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1 2020-03-16 13:34:59.555678: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session 2020-03-16 13:34:59.564958: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:34:59.568464: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:34:59.570953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:34:59.572234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-16 13:34:59.574515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-03-16 13:34:59.575663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2020-03-16 13:34:59.577064: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6351 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1) 2020-03-16 13:34:59.736086: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize 2020-03-16 13:34:59.738817: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: Graph size after: 2183 nodes (1811), 5175 edges (4800), time = 79.052ms. 2020-03-16 13:34:59.741706: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: function_optimizer did nothing. time = 1.372ms. 2020-03-16 13:35:05.101976: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:35:05.106896: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:35:05.110194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:35:05.112845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-16 13:35:05.115719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-03-16 13:35:05.117089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2020-03-16 13:35:05.119435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6351 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1) WARNING:tensorflow:From D:\work\cv\others\tensorflow-onnx\tf2onnx\tf_loader.py:305: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph 2020-03-16 13:35:05,807 - WARNING - tensorflow: From D:\work\cv\others\tensorflow-onnx\tf2onnx\tf_loader.py:305: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph 2020-03-16 13:35:06.132003: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1 2020-03-16 13:35:06.134612: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session 2020-03-16 13:35:06.136366: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:35:06.138836: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:35:06.142093: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:35:06.143392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-16 13:35:06.145154: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-03-16 13:35:06.146189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2020-03-16 13:35:06.147499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6351 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1) 2020-03-16 13:35:11.442378: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize 2020-03-16 13:35:11.444175: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] constant folding: Graph size after: 1378 nodes (-358), 3856 edges (-716), time = 3517.05ms. 2020-03-16 13:35:11.446392: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: function_optimizer did nothing. time = 19.883ms. 2020-03-16 13:35:11.447977: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] constant folding: Graph size after: 1378 nodes (0), 3856 edges (0), time = 1067.97095ms. 2020-03-16 13:35:11.449778: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: function_optimizer did nothing. time = 28.142ms. 2020-03-16 13:35:12,329 - INFO - tf2onnx: inputs: dict_keys(['input:0']) 2020-03-16 13:35:12,330 - INFO - tf2onnx: outputs: dict_keys(['Identity:0', 'Identity_1:0', 'Identity_2:0', 'Identity_3:0']) 2020-03-16 13:35:13.110494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095 pciBusID: 0000:01:00.0 2020-03-16 13:35:13.114622: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check. 2020-03-16 13:35:13.117316: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2020-03-16 13:35:13.118835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-16 13:35:13.121375: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-03-16 13:35:13.122781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2020-03-16 13:35:13.124682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6351 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1) 2020-03-16 13:35:13,127 - INFO - tf2onnx.tfonnx: Using tensorflow=2.0.1, onnx=1.6.0, tf2onnx=1.6.0/f48281 2020-03-16 13:35:13,127 - INFO - tf2onnx.tfonnx: Using opset <onnx, 8> 2020-03-16 13:35:13,149 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization/FusedBatchNormV3:5 2020-03-16 13:35:13,149 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_1/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_1/FusedBatchNormV3:5 2020-03-16 13:35:13,150 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_2/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_2/FusedBatchNormV3:5 2020-03-16 13:35:13,150 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_3/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_3/FusedBatchNormV3:5 2020-03-16 13:35:13,150 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_4/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_4/FusedBatchNormV3:5 2020-03-16 13:35:13,150 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_5/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_5/FusedBatchNormV3:5 2020-03-16 13:35:13,150 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_6/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_6/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_7/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_7/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_8/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_8/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_9/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_9/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_10/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_10/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_11/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_11/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_12/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_12/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_13/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_13/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_14/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_14/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_15/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_15/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_16/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_16/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_17/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_17/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_18/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_18/FusedBatchNormV3:5 2020-03-16 13:35:13,151 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_19/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_19/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_20/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_20/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_21/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_21/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_22/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_22/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_23/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_23/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_24/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_24/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_25/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_25/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_26/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_26/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_27/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_27/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_28/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_28/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_29/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_29/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_30/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_30/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_31/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_31/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_32/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_32/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_33/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_33/FusedBatchNormV3:5 2020-03-16 13:35:13,152 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_34/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_34/FusedBatchNormV3:5 2020-03-16 13:35:13,153 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_35/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_35/FusedBatchNormV3:5 2020-03-16 13:35:13,153 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_36/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_36/FusedBatchNormV3:5 2020-03-16 13:35:13,153 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_37/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_37/FusedBatchNormV3:5 2020-03-16 13:35:13,153 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_38/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_38/FusedBatchNormV3:5 2020-03-16 13:35:13,176 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_39/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_39/FusedBatchNormV3:5 2020-03-16 13:35:13,177 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_40/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_40/FusedBatchNormV3:5 2020-03-16 13:35:13,177 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_41/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_41/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_42/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_42/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_43/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_43/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_44/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_44/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_45/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_45/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_46/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_46/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_47/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_47/FusedBatchNormV3:5 2020-03-16 13:35:13,178 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_48/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_48/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_49/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_49/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_50/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_50/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_51/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_darknet/batch_normalization_51/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_52/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_52/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_53/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_53/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_54/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_54/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_55/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_55/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_56/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_0/batch_normalization_56/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_output_0/batch_normalization_57/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_output_0/batch_normalization_57/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_58/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_58/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_59/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_59/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_60/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_60/FusedBatchNormV3:5 2020-03-16 13:35:13,179 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_61/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_61/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_62/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_62/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_63/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_1/batch_normalization_63/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_output_1/batch_normalization_64/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_output_1/batch_normalization_64/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_65/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_65/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_66/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_66/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_67/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_67/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_68/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_68/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_69/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_69/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_70/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_conv_2/batch_normalization_70/FusedBatchNormV3:5 2020-03-16 13:35:13,180 - WARNING - tf2onnx.shape_inference: Cannot infer shape for StatefulPartitionedCall/yolov3/yolo_output_2/batch_normalization_71/FusedBatchNormV3: StatefulPartitionedCall/yolov3/yolo_output_2/batch_normalization_71/FusedBatchNormV3:5 2020-03-16 13:35:24,116 - VERBOSE - tf2onnx.tfonnx: Mapping TF node to ONNX node(s) 2020-03-16 13:35:26,084 - ERROR - tf2onnx.tfonnx: Failed to convert node StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/resize/ResizeNearestNeighbor OP=Upsample Name=StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/resize/ResizeNearestNeighbor Inputs: StatefulPartitionedCall/yolov3/yolo_conv_1/leaky_re_lu_58/LeakyRelu:0=LeakyRelu, [-1, -1, -1, 256], 1 StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/mul:0=Mul, [2], 6 Outpus: StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/resize/ResizeNearestNeighbor:0=[-1, -1, -1, 256], 1 Traceback (most recent call last): File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tfonnx.py", line 266, in tensorflow_onnx_mapping func(g, node, kwargs) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\onnx_opset\nn.py", line 658, in version_7 target_shape = node.inputs[1].get_tensor_value() File "D:\work\cv\others\tensorflow-onnx\tf2onnx\graph.py", line 266, in get_tensor_value raise ValueError("get tensor value: {} must be Const".format(self.name)) ValueError: get tensor value: StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/mul must be Const 2020-03-16 13:35:26,427 - ERROR - tf2onnx.tfonnx: Failed to convert node StatefulPartitionedCall/yolov3/yolo_conv_2/up_sampling2d_1/resize/ResizeNearestNeighbor OP=Upsample Name=StatefulPartitionedCall/yolov3/yolo_conv_2/up_sampling2d_1/resize/ResizeNearestNeighbor Inputs: StatefulPartitionedCall/yolov3/yolo_conv_2/leaky_re_lu_65/LeakyRelu:0=LeakyRelu, [-1, -1, -1, 128], 1 StatefulPartitionedCall/yolov3/yolo_conv_2/up_sampling2d_1/mul:0=Mul, [2], 6 Outpus: StatefulPartitionedCall/yolov3/yolo_conv_2/up_sampling2d_1/resize/ResizeNearestNeighbor:0=[-1, -1, -1, 128], 1 Traceback (most recent call last): File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tfonnx.py", line 266, in tensorflow_onnx_mapping func(g, node, kwargs) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\onnx_opset\nn.py", line 658, in version_7 target_shape = node.inputs[1].get_tensor_value() File "D:\work\cv\others\tensorflow-onnx\tf2onnx\graph.py", line 266, in get_tensor_value raise ValueError("get tensor value: {} must be Const".format(self.name)) ValueError: get tensor value: StatefulPartitionedCall/yolov3/yolo_conv_2/up_sampling2d_1/mul must be Const 2020-03-16 13:35:26,724 - ERROR - tf2onnx.tfonnx: Tensorflow op [StatefulPartitionedCall/yolov3/yolo_nms/combined_non_max_suppression/CombinedNonMaxSuppression: CombinedNonMaxSuppression] is not supported 2020-03-16 13:35:26,725 - ERROR - tf2onnx.tfonnx: Unsupported ops: Counter({'CombinedNonMaxSuppression': 1}) Traceback (most recent call last): File "D:\software\conda_envs\yolov3-tf2-gpu\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "D:\software\conda_envs\yolov3-tf2-gpu\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\convert.py", line 169, in main() File "D:\work\cv\others\tensorflow-onnx\tf2onnx\convert.py", line 153, in main inputs_as_nchw=args.inputs_as_nchw) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tfonnx.py", line 485, in process_tf_graph raise exceptions[0] File "D:\work\cv\others\tensorflow-onnx\tf2onnx\tfonnx.py", line 266, in tensorflow_onnx_mapping func(g, node, **kwargs) File "D:\work\cv\others\tensorflow-onnx\tf2onnx\onnx_opset\nn.py", line 658, in version_7 target_shape = node.inputs[1].get_tensor_value() File "D:\work\cv\others\tensorflow-onnx\tf2onnx\graph.py", line 266, in get_tensor_value raise ValueError("get tensor value: {} must be Const".format(self.name)) ValueError: get tensor value: StatefulPartitionedCall/yolov3/yolo_conv_1/up_sampling2d/mul must be Const

jignparm commented 4 years ago

It looks like there are a couple of issues.

The unsupported operator will need to be supported in Onnx or else composed using a combination of other operators.

Is this a YoloV3 model? How is this model different from the YoloV3 Onnx model here --> https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/yolov3

guschmue commented 4 years ago

by default we use opset 8 but there are a bunch of changes around Upsample after opset 8 which should help. Can you try with --opset 11 ?

jackyvr commented 4 years ago

It looks like there are a couple of issues.

  • An input into ResizeNearestNeighbor is not a constant,
  • the operator CombinedNonMaxSuppression is not supported in Onnx.

The unsupported operator will need to be supported in Onnx or else composed using a combination of other operators.

Is this a YoloV3 model? How is this model different from the YoloV3 Onnx model here --> https://github.com/onnx/models/blob/master/vision/object_detection_segmentation/yolov3

Thanks @jignparm . Yes, it is Yolov3. I built a TF2 yolov3 model with my own custom data.

guschmue commented 4 years ago

The ResizeNearestNeighbor should not be an issue on opset 11. CombinedNonMaxSuppression - yes, we don't map this. Guess that comes from tf.image.combined_non_max_suppression. We can check if we can map this to onnx.

rkpandya commented 4 years ago

I am trying to convert a tensorflow model to onnx, where the conversion hangs up. Related to the closed issue #739 (https://github.com/onnx/tensorflow-onnx/issues/739)

Have updated tensorflow-gpu to the latest and tf2onnx to the latest 1.5.5.

Using a model based on DeepLab V3+, using opset 11.

jignparm commented 4 years ago

@rkpandya -- please see response at #739. The comment doesn't seem relevant to this thread however :)

jackyvr commented 4 years ago

@guschmue and @jignparm , with --opset 11, I am able to generate .onnx output. I haven't tested the output though, but I assume it should work. Thanks.

I will close the threshold after verification.

edurenye commented 4 years ago

I don't understand, why it works with --opset 11 if CombinedNonMaxSuppression is not supported by Onnx?

guschmue commented 4 years ago

There is a tensorflow op CombinedNonMaxSuppression which we don't implement. I have not seen this with Yolov3, Normally we see NonMaxSuppressionV[2,3,4,5] which we implement. I need to check when tensorflow uses CombinedNonMaxSuppression and when yolov3 would trigger it. opset should not matter. Maybe the tensorflow version matters.

edurenye commented 4 years ago

I'm using TF 2.2, there is any way to find which layer is using the CombinedNonMaxSuppression op so I can try to replace it?

edurenye commented 4 years ago

Found it: https://github.com/zzh8829/yolov3-tf2/blob/d63a9921a42dce04cd74ddb529926f9ff81bc41a/yolov3_tf2/models.py#L191

But I'm not really sure how to replace it for tf.image.non_max_suppression_padded or tf.image.non_max_suppression_with_scores

guschmue commented 4 years ago

Ah. It does padding, clips and returns the gathered boxes. I think we can do this in tf2onnx but it will take us a little to get to it. The replacement could be something like:

nms_index = tf.image.non_max_suppression(coors, scores, max_boxes_tensor, iou_threshold=iou_threshold)
bboxes = tf.concat([tf.gather(coors, nms_index), 
                    tf.gather(scores[:, np.newaxis], nms_index),
                    tf.gather(classes[:, np.newaxis], nms_index)], axis=-1)
edurenye commented 4 years ago

Thank you very much for your help and for answering so quick.

And I'm sorry because I'm still learning and there are many things I don't understand yet. I don't get it, why use tf.concat? I need to return the elements as individually. And there is expected the valid_detections as a return value, that I'm not sure how to get.

Shouldn't it be something similar to this?

nms_index = tf.image.non_max_suppression(
    boxes = bbox,
    scores = scores,
    max_output_size = FLAGS.yolo_max_boxes,
    iou_threshold = FLAGS.yolo_iou_threshold,
    score_threshold = FLAGS.yolo_score_threshold
)
boxes = tf.gather(bbox, nms_index)
scores = tf.gather(scores, nms_index)
classes = tf.gather(classes, nms_index)
valid_detections = tf.size(nms_index)
guschmue commented 4 years ago

yeap, something like this should work.

edurenye commented 4 years ago

It trains fine, but when I try to predict it gives me the following error:

File "/root/miniconda3/envs/yolov3-tf2-gpu/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1657, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 2 but is rank 3 for '{{node yolo_nms/non_max_suppression/NonMaxSuppressionV3}} = NonMaxSuppressionV3[T=DT_FLOAT, T_threshold=DT_FLOAT](yolo_nms/concat, yolo_nms/mul, yolo_nms/non_max_suppression/NonMaxSuppressionV3/max_output_size, yolo_nms/non_max_suppression/iou_threshold, yolo_nms/non_max_suppression/score_threshold)' with input shapes: [?,?,?], [?,?,?], [], [], [].

Any idea of what I've done wrong?

edurenye commented 4 years ago

Replacing:

    boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
        boxes=tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)),
        scores=tf.reshape(
            scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])),
        max_output_size_per_class=FLAGS.yolo_max_boxes,
        max_total_size=FLAGS.yolo_max_boxes,
        iou_threshold=FLAGS.yolo_iou_threshold,
        score_threshold=FLAGS.yolo_score_threshold
    )

For:

    all_boxes = tf.reshape(bbox, (-1, 4))
    all_scores = tf.reshape(scores, (-1, tf.shape(scores)[-1]))
    num_classes = classes
    my_structure = [{'boxes': [], 'scores': [], 'classes': []}] * num_classes
    all_classes = tf.argmax(all_scores, axis=1)
    all_scores = tf.reduce_max(all_scores, axis=1)
    for i in range(len(all_classes)):
        c = all_classes[i]
        my_structure[c]['boxes'].append(all_boxes[i])
        my_structure[c]['scores'].append(all_scores[i])
        my_structure[c]['classes'].append(tf.cast(c, tf.float32))
    all_nms_index = []
    boxes = [[]]
    scores = [[]]
    classes = [[]]
    for c in range(num_classes):
        nms_index = tf.image.non_max_suppression(
            boxes = my_structure[c]['boxes'],
            scores = my_structure[c]['scores'],
            max_output_size = FLAGS.yolo_max_boxes,
            iou_threshold = FLAGS.yolo_iou_threshold,
            score_threshold = FLAGS.yolo_score_threshold
        )
        all_nms_index.extend(nms_index)
        boxes[0].extend(tf.gather(my_structure[c]['boxes'], nms_index))
        scores[0].extend(tf.gather(my_structure[c]['scores'], nms_index))
        classes[0].extend(tf.gather(my_structure[c]['classes'], nms_index))
    boxes = tf.convert_to_tensor(boxes, dtype=tf.float32)
    scores = tf.convert_to_tensor(scores, dtype=tf.float32)
    classes = tf.convert_to_tensor(classes, dtype=tf.float32)
    valid_detections = tf.convert_to_tensor([tf.size(all_nms_index)], dtype=tf.int32)

Works fine, but yes, I'm sure I can simplify that hell of code.

neo- commented 4 years ago

I was trying to apply your solution. I got an error on the line len(all_classes)

which says: len is not well defined for symbolic Tensors. (yolo_nms/ArgMax:0) Please call x.shape rather than len(x) for shape information.

Here is the code how it looks like after replacing:

def yolo_nms(outputs, anchors, masks, classes):

b, c, t = [], [], []
for o in outputs:
    b.append(tf.reshape(o[0], (tf.shape(o[0])[0], -1, tf.shape(o[0])[-1])))
    c.append(tf.reshape(o[1], (tf.shape(o[1])[0], -1, tf.shape(o[1])[-1])))
    t.append(tf.reshape(o[2], (tf.shape(o[2])[0], -1, tf.shape(o[2])[-1])))

bbox = tf.concat(b, axis=1)
confidence = tf.concat(c, axis=1)
class_probs = tf.concat(t, axis=1)

scores = confidence * class_probs

all_boxes = tf.reshape(bbox, (-1, 4))
all_scores = tf.reshape(scores, (-1, tf.shape(scores)[-1]))
num_classes = classes
my_structure = [{'boxes': [], 'scores': [], 'classes': []}] * num_classes
all_classes = tf.argmax(all_scores, axis=1)
all_scores = tf.reduce_max(all_scores, axis=1)
for i in range(len(all_classes)):
    c = all_classes[i]
    my_structure[c]['boxes'].append(all_boxes[i])
    my_structure[c]['scores'].append(all_scores[i])
    my_structure[c]['classes'].append(tf.cast(c, tf.float32))
all_nms_index = []
boxes = [[]]
scores = [[]]
classes = [[]]
for c in range(num_classes):
    nms_index = tf.image.non_max_suppression(
        boxes=my_structure[c]['boxes'],
        scores=my_structure[c]['scores'],
        max_output_size=FLAGS.yolo_max_boxes,
        iou_threshold=FLAGS.yolo_iou_threshold,
        score_threshold=FLAGS.yolo_score_threshold
    )
    all_nms_index.extend(nms_index)
    boxes[0].extend(tf.gather(my_structure[c]['boxes'], nms_index))
    scores[0].extend(tf.gather(my_structure[c]['scores'], nms_index))
    classes[0].extend(tf.gather(my_structure[c]['classes'], nms_index))
boxes = tf.convert_to_tensor(boxes, dtype=tf.float32)
scores = tf.convert_to_tensor(scores, dtype=tf.float32)
classes = tf.convert_to_tensor(classes, dtype=tf.float32)
valid_detections = tf.convert_to_tensor([tf.size(all_nms_index)], dtype=tf.int32)
return boxes, scores, classes, valid_detections

`

May I know where I got it wrong?

edurenye commented 4 years ago

That worked for me, but you can try to use tf.size() instead of len() not sure if range() will like that as an input though, in case it fails you can try to cast it to int type.

marcelvanworkum commented 4 years ago

@edurenye I've run into the same issue as @neo- when running your fix. I tried your suggestion of tf.size() but as you correctly guessed, range did not like the tensor input. I think you must have a slightly different environment configuration to @neo- and I. Would you be able to share your complete models.py and perhaps your requirements.txt.

I notice that you solved a similar issue in keras-rl2 - perhaps you could help shed some light on how to fix this issue.

Fay-Hu commented 4 years ago

@edurenye I've run into the same issue as @neo- when running your fix. I tried your suggestion of tf.size() but as you correctly guessed, range did not like the tensor input. I think you must have a slightly different environment configuration to @neo- and I. Would you be able to share your complete models.py and perhaps your requirements.txt.

I notice that you solved a similar issue in keras-rl2 - perhaps you could help shed some light on how to fix this issue.

Hi, do you solve it now? I've also had the same problem.

edurenye commented 4 years ago

@marcelvanworkum Later it failed again, I think that time I tried the code did not get triggered, that is why I thought it worked, but it really did not. At the end I removed all that code from the conversion and added it outside the model.

marcelvanworkum commented 4 years ago

Thanks for your response @edurenye. I've also gone with a similar approach, simply just performing the NMS in code.

guschmue commented 4 years ago

I assume this is no longer a issue.

rajan780 commented 3 years ago

Replacing:

    boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
        boxes=tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)),
        scores=tf.reshape(
            scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])),
        max_output_size_per_class=FLAGS.yolo_max_boxes,
        max_total_size=FLAGS.yolo_max_boxes,
        iou_threshold=FLAGS.yolo_iou_threshold,
        score_threshold=FLAGS.yolo_score_threshold
    )

For:

    all_boxes = tf.reshape(bbox, (-1, 4))
    all_scores = tf.reshape(scores, (-1, tf.shape(scores)[-1]))
    num_classes = classes
    my_structure = [{'boxes': [], 'scores': [], 'classes': []}] * num_classes
    all_classes = tf.argmax(all_scores, axis=1)
    all_scores = tf.reduce_max(all_scores, axis=1)
    for i in range(len(all_classes)):
        c = all_classes[i]
        my_structure[c]['boxes'].append(all_boxes[i])
        my_structure[c]['scores'].append(all_scores[i])
        my_structure[c]['classes'].append(tf.cast(c, tf.float32))
    all_nms_index = []
    boxes = [[]]
    scores = [[]]
    classes = [[]]
    for c in range(num_classes):
        nms_index = tf.image.non_max_suppression(
            boxes = my_structure[c]['boxes'],
            scores = my_structure[c]['scores'],
            max_output_size = FLAGS.yolo_max_boxes,
            iou_threshold = FLAGS.yolo_iou_threshold,
            score_threshold = FLAGS.yolo_score_threshold
        )
        all_nms_index.extend(nms_index)
        boxes[0].extend(tf.gather(my_structure[c]['boxes'], nms_index))
        scores[0].extend(tf.gather(my_structure[c]['scores'], nms_index))
        classes[0].extend(tf.gather(my_structure[c]['classes'], nms_index))
    boxes = tf.convert_to_tensor(boxes, dtype=tf.float32)
    scores = tf.convert_to_tensor(scores, dtype=tf.float32)
    classes = tf.convert_to_tensor(classes, dtype=tf.float32)
    valid_detections = tf.convert_to_tensor([tf.size(all_nms_index)], dtype=tf.int32)

Works fine, but yes, I'm sure I can simplify that hell of code.

Hi @edurenye I replaced the existing code with this code and i am also getting same error as @neo- is getting. is this issue solved? and any suggestions how we can use tf.image.non_max_suppression_with_scores instead of tf.image.combined_non_max_suppression.

edurenye commented 3 years ago

Hi @rajan780, As I said in https://github.com/onnx/tensorflow-onnx/issues/847#issuecomment-645245677 I never got it working, I had to work around it. Now that project I was working on is closed and I haven't tried it again, sorry.

rajan780 commented 3 years ago

Hi @rajan780, As I said in #847 (comment) I never got it working, I had to work around it. Now that project I was working on is closed and I haven't tried it again, sorry.

Thanks for your response @edurenye.

rajan780 commented 3 years ago

Hi @rajan780, As I said in #847 (comment) I never got it working, I had to work around it. Now that project I was working on is closed and I haven't tried it again, sorry.

Hi @edurenye I made changes in code. It is working now.

Replacing: boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression( boxes=tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)), scores=tf.reshape( scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])), max_output_size_per_class=FLAGS.yolo_max_boxes, max_total_size=FLAGS.yolo_max_boxes, iou_threshold=FLAGS.yolo_iou_threshold, score_threshold=FLAGS.yolo_score_threshold )

For:

scores = tf.reshape(scores,[-1])
bbox = tf.reshape(bbox,(-1,4))

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(
    boxes=bbox,
    scores=scores,
    max_output_size=FLAGS.yolo_max_boxes,
    iou_threshold=FLAGS.yolo_iou_threshold,
    score_threshold=FLAGS.yolo_score_threshold,
    soft_nms_sigma=0.5
)

num_valid_nms_boxes = tf.shape(selected_indices)[0]
selected_indices = tf.concat([selected_indices,tf.zeros(FLAGS.yolo_max_boxes-num_valid_nms_boxes, tf.int32)], 0)
selected_scores = tf.concat([selected_scores,tf.zeros(FLAGS.yolo_max_boxes-num_valid_nms_boxes,tf.float32)], -1)
boxes=tf.gather(bbox, selected_indices)
scores=selected_scores
valid_detections=num_valid_nms_boxes

return boxes, scores, valid_detections

VeeranjaneyuluToka commented 3 years ago

@rajan780 , thanks for sharing the working solutions for tf.image.non_max_suppression_with_scores function. I am facing same issue and i have couple of things to clarify before i proceed.

In original code, it returns 4 values/tensors (boxes, scores, classes, valid_detections) but your modified code returns only boxes, scores and valid_detections, how to get classes also here? And do we need to retrain the model with this change or if we have checkpoints of trained model, you can create a savedmodel with inference by loading weights from checkpoints and convert to onnx?

VeeranjaneyuluToka commented 3 years ago

@rajan780 , i tried your solution and still it is not getting converted, here is the log Command used: python -m tf2onnx.convert --saved-model /tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/20210209-214418_sm_new_nms/ --output /tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/model.onnx

2021-02-11 19:01:53,057 - INFO - Using tensorflow=2.4.0, onnx=1.8.1, tf2onnx=1.8.2/05babf 2021-02-11 19:01:53,057 - INFO - Using opset <onnx, 9> 2021-02-11 19:01:56,608 - INFO - Computed 0 values for constant folding 2021-02-11 19:02:06,229 - ERROR - Tensorflow op [StatefulPartitionedCall/yolov3/yolo_nms/non_max_suppression_with_scores/NonMaxSuppressionV5: NonMaxSuppressionV5] is not supported 2021-02-11 19:02:06,231 - ERROR - Unsupported ops: Counter({'NonMaxSuppressionV5': 1}) 2021-02-11 19:02:06,280 - INFO - Optimizing ONNX model 2021-02-11 19:02:10,112 - INFO - After optimization: BatchNormalization -72 (72->0), Cast -35 (97->62), Const -401 (570->169), Gather +5 (1->6), Identity -19 (31->12), Mul -3 (18->15), Reshape -1 (28->27), Shape -14 (32->18), Squeeze -24 (31->7), Sub -1 (11->10), Transpose -294 (298->4), Unsqueeze -42 (57->15) 2021-02-11 19:02:10,195 - INFO - 2021-02-11 19:02:10,195 - INFO - Successfully converted TensorFlow model

The below command helped me to resolve the above issue (setting opset to 11): python -m tf2onnx.convert --saved-model /tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/20210209-214418_sm_new_nms/ --output tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/model.onnx --opset 11

but i could not convert from onnx to trt when i use opset 11, when i downgrade opset version to 10, i am getting the below error

ERROR: builtin_op_importers.cpp:2651 In function importResize: [8] Assertion failed: scales.is_weights() && "Resize scales must be an initializer!"

rajan780 commented 3 years ago

@rajan780 , thanks for sharing the working solutions for tf.image.non_max_suppression_with_scores function. I am facing same issue and i have couple of things to clarify before i proceed.

In original code, it returns 4 values/tensors (boxes, scores, classes, valid_detections) but your modified code returns only boxes, scores and valid_detections, how to get classes also here? And do we need to retrain the model with this change or if we have checkpoints of trained model, you can create a savedmodel with inference by loading weights from checkpoints and convert to onnx?

Hi @VeeranjaneyuluToka I implemented this api in the training repo and it's working but i did not check for conversion. sorry for that. and with respect to class, i train only for one class so for one class we can return like (classes = tf.reshape(classes,[1])) and return classes but in my case i did not want so i did not return.

Note: we can do something like this and i check this in the training repo it is working but not for conversion:

        scores = tf.reshape(scores,[-1])
        bbox = tf.reshape(bbox,(-1,4))
        classes = tf.reshape(classes,[1])

        selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(
            boxes=bbox,
            scores=scores,
            max_output_size=FLAGS.yolo_max_boxes,
            iou_threshold=FLAGS.yolo_iou_threshold,
            score_threshold=FLAGS.yolo_score_threshold,
            soft_nms_sigma=0.5
        )
        num_valid_nms_boxes = tf.shape(selected_indices)[0]
        selected_indices = tf.concat([selected_indices,tf.zeros(FLAGS.yolo_max_boxes-num_valid_nms_boxes, tf.int32)], -1)
        selected_scores = tf.concat([selected_scores,tf.zeros(FLAGS.yolo_max_boxes-num_valid_nms_boxes,tf.float32)], -1)
        classes = tf.concat([classes,tf.ones(FLAGS.yolo_max_boxes-1, tf.int32)], 0)            

        boxes=tf.gather(bbox, selected_indices)
        boxes = tf.expand_dims(boxes,axis=0)
        scores=selected_scores
        scores = tf.expand_dims(scores,axis=0)  
        classes = tf.expand_dims(classes,axis=0)
        valid_detections=num_valid_nms_boxes
        valid_detections = tf.expand_dims(valid_detections,axis=0)

    return boxes, scores, classes, valid_detections
VeeranjaneyuluToka commented 3 years ago

Hi, Thanks for reply! I just tried to run the inference with this change and not able to run inference. Have you run inference with out errors? I am getting below errors while running inference InvalidArgumentError: scores has incompatible shape [Op:NonMaxSuppressionV5]

As i mentioned i have trained already a model and saved checkpoints while running inference i changed as you suggested above but getting the above error. When i check the shape of scores and boxes, they are as follows

Original score shape (before you reshape): (1, 10647, 5) , there are 5 classes in my dataset. do not understand why you are reshaping as below scores = tf.reshape(scores,[-1]) when i check the documentation of tf.image.non_max_suppression_with_scores, it says as below about scores parameter A 1-D float Tensor of shape [num_boxes] representing a single score corresponding to each box (each row of boxes) I believe it means that we just need to pass all the boxes scores (with out reshaping it) is not it?

Scores shape before pass to the nms:(53235,) boxes shape before pass to the nms:(10647, 4) wondering why should we multiply scores shape to (10647x5) and reshape the tensor?

I believe you might have tested inference?

rajan780 commented 3 years ago

Hi, Thanks for reply! I just tried to run the inference with this change and not able to run inference. Have you run inference with out errors? I am getting below errors while running inference InvalidArgumentError: scores has incompatible shape [Op:NonMaxSuppressionV5]

As i mentioned i have trained already a model and saved checkpoints while running inference i changed as you suggested above but getting the above error. When i check the shape of scores and boxes, they are as follows

Original score shape (before you reshape): (1, 10647, 5) , there are 5 classes in my dataset. do not understand why you are reshaping as below scores = tf.reshape(scores,[-1]) when i check the documentation of tf.image.non_max_suppression_with_scores, it says as below about scores parameter A 1-D float Tensor of shape [num_boxes] representing a single score corresponding to each box (each row of boxes) I believe it means that we just need to pass all the boxes scores (with out reshaping it) is not it?

Scores shape before pass to the nms:(53235,) boxes shape before pass to the nms:(10647, 4) wondering why should we multiply scores shape to (10647x5) and reshape the tensor?

I believe you might have tested inference?

@VeeranjaneyuluToka Yes, i am able to run inference without errors. actually in my case i am having only one class. So for example: my scores without reshape has shape(1,10647,1) . so after reshape (scores = tf.reshape(scores,[-1])) shape will be (10647). But in your case you have 5 classes but non_max_suppression_with_scores api want A 1-D float Tensor of shape [num_boxes] representing a single score corresponding to each box (each row of boxes). so you have to pass scores with shape(10647) this reshape you have to modify accordingly. as you are reshaping with scores = tf.reshape(scores,[-1]) it will make the shape(53235) which is wrong you have to pass with shape(10647).

VeeranjaneyuluToka commented 3 years ago

@rajan780 , thanks for your reply again! I just did as below to pass score array as 1d in my case

sscores = tf.squeeze(scores, axis=0) #squeezing from (1, 10647, 5) to (10647, 5)
nscores = tf.gather(sscores, [0], axis=1) #get first column elements
dscores = tf.reshape(nscores, [-1]) #reshape from (10647, 1) to (10647)
bbox = tf.reshape(bbox,(-1,4))

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, **scores=dscores**, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

Anything wrong you can notice in this, i am not getting any errors with the above change but am not getting any predictions.

rajan780 commented 3 years ago

@rajan780 , thanks for your reply again! I just did as below to pass score array as 1d in my case

sscores = tf.squeeze(scores, axis=0) #squeezing from (1, 10647, 5) to (10647, 5)
nscores = tf.gather(sscores, [0], axis=1) #get first column elements
dscores = tf.reshape(nscores, [-1]) #reshape from (10647, 1) to (10647)
bbox = tf.reshape(bbox,(-1,4))

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, **scores=dscores**, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

Anything wrong you can notice in this, i am not getting any errors with the above change but am not getting any predictions.

Hi @VeeranjaneyuluToka please try this and check: sscores = tf.squeeze(scores, axis=0) #squeezing from (1, 10647, 5) to (10647, 5) dscores= tf.reduce_max(sscores,[1])#this will have shape(10647) selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, scores=dscores, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

VeeranjaneyuluToka commented 3 years ago

Hi @rajan780 , Thanks for your suggestion again!

Yes, it works wrt bounding box predictions but predicted class names are not correct. Class names are quite different compare to tf.image.Combined_non_max_suppression code. Do you suspect any reason for this?

rajan780 commented 3 years ago

Hi @rajan780 , Thanks for your suggestion again!

Yes, it works wrt bounding box predictions but predicted class names are not correct. Class names are quite different compare to tf.image.Combined_non_max_suppression code. Do you suspect any reason for this?

Hi @VeeranjaneyuluToka combined_non_max_suppressio api will return classes also for each box but non_max_suppression_with_score will not return classes. So we have to do some workaround for that: Please try and check this but i am not very sure if that will work or not because in my case i am having only one class so i did not worry more about classes.

sscores = tf.squeeze(scores, axis=0) #squeezing from (1, 10647, 5) to (10647, 5) dscores= tf.reduce_max(sscores,[1])#this will have shape(10647) classes = tf.argmax(sscores,1)#this will have shape(10647) selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, scores=dscores, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

VeeranjaneyuluToka commented 3 years ago

Hi Again, no that is not working, it gives error as below InvalidArgumentError: Expected dimension in the range [0, 0), but got 1 [Op:ArgMax]

rajan780 commented 3 years ago

Hi Again, no that is not working, it gives error as below InvalidArgumentError: Expected dimension in the range [0, 0), but got 1 [Op:ArgMax]

Thanks for your response @VeeranjaneyuluToka currently, i am also not having an idea how to know about classes. i will try to fix this and update you.can you share your piece of code also?

and if you have any update from your side please let me know:)

VeeranjaneyuluToka commented 3 years ago

Hi, Here is the complete yolo_nms function that i have been using. (mostly modified based on your suggestions)

def yolo_nms(outputs, anchors, masks, classes):

b, c, t = [], [], []

for o in outputs:
    b.append(tf.reshape(o[0], (tf.shape(o[0])[0], -1, tf.shape(o[0])[-1])))
    c.append(tf.reshape(o[1], (tf.shape(o[1])[0], -1, tf.shape(o[1])[-1])))
    t.append(tf.reshape(o[2], (tf.shape(o[2])[0], -1, tf.shape(o[2])[-1])))

bbox = tf.concat(b, axis=1)
confidence = tf.concat(c, axis=1)
class_probs = tf.concat(t, axis=1)
scores = confidence * class_probs

'''
boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
        boxes = tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)),
        scores = tf.reshape(scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])),
        max_output_size_per_class=100, max_total_size=100, iou_threshold=0.5, score_threshold=0.5)

return boxes, scores, classes, valid_detections'''

sscores = tf.squeeze(scores, axis=0)
dscores = tf.reduce_max(sscores, [1]) #this will have shape(10647)
bbox = tf.reshape(bbox, (-1, 4))
#classes = tf.argmax(classes, 1)
classes = tf.reshape(classes,[1])

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, scores=dscores, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

num_valid_nms_boxes = tf.shape(selected_indices)[0]
selected_indices = tf.concat([selected_indices, tf.zeros(100-num_valid_nms_boxes, tf.int32)], -1)
selected_scores = tf.concat([selected_scores, tf.zeros(100-num_valid_nms_boxes, tf.float32)], -1)
classes = tf.concat([classes, tf.ones(100 - 1, tf.int32)], 0)

boxes = tf.gather(bbox, selected_indices)
boxes = tf.expand_dims(boxes, axis=0)
scores = selected_scores
scores = tf.expand_dims(scores, axis=0)
classes = tf.expand_dims(classes, axis=0)
valid_detections = num_valid_nms_boxes
valid_detections = tf.expand_dims(valid_detections, axis=0)

return boxes, scores, classes, valid_detections
rajan780 commented 3 years ago

Hi, Here is the complete yolo_nms function that i have been using. (mostly modified based on your suggestions)

def yolo_nms(outputs, anchors, masks, classes):

b, c, t = [], [], []

for o in outputs:
    b.append(tf.reshape(o[0], (tf.shape(o[0])[0], -1, tf.shape(o[0])[-1])))
    c.append(tf.reshape(o[1], (tf.shape(o[1])[0], -1, tf.shape(o[1])[-1])))
    t.append(tf.reshape(o[2], (tf.shape(o[2])[0], -1, tf.shape(o[2])[-1])))

bbox = tf.concat(b, axis=1)
confidence = tf.concat(c, axis=1)
class_probs = tf.concat(t, axis=1)
scores = confidence * class_probs

'''
boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
        boxes = tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)),
        scores = tf.reshape(scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])),
        max_output_size_per_class=100, max_total_size=100, iou_threshold=0.5, score_threshold=0.5)

return boxes, scores, classes, valid_detections'''

sscores = tf.squeeze(scores, axis=0)
dscores = tf.reduce_max(sscores, [1]) #this will have shape(10647)
bbox = tf.reshape(bbox, (-1, 4))
#classes = tf.argmax(classes, 1)
classes = tf.reshape(classes,[1])

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, scores=dscores, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

num_valid_nms_boxes = tf.shape(selected_indices)[0]
selected_indices = tf.concat([selected_indices, tf.zeros(100-num_valid_nms_boxes, tf.int32)], -1)
selected_scores = tf.concat([selected_scores, tf.zeros(100-num_valid_nms_boxes, tf.float32)], -1)
classes = tf.concat([classes, tf.ones(100 - 1, tf.int32)], 0)

boxes = tf.gather(bbox, selected_indices)
boxes = tf.expand_dims(boxes, axis=0)
scores = selected_scores
scores = tf.expand_dims(scores, axis=0)
classes = tf.expand_dims(classes, axis=0)
valid_detections = num_valid_nms_boxes
valid_detections = tf.expand_dims(valid_detections, axis=0)

return boxes, scores, classes, valid_detections

Hi @VeeranjaneyuluToka Please comment these two lines from your code:

classes = tf.argmax(classes, 1)

classes = tf.reshape(classes,[1])

And instead of above mentioned two lines change to: classes = tf.argmax(sscores,1)#this will have shape(10647)

After that comment that line also: classes = tf.concat([classes, tf.ones(100 - 1, tf.int32)], 0)

And before that line: classes = tf.expand_dims(classes, axis=0) Add this: classes = tf.gather(classes,selected_indices)

rajan780 commented 3 years ago

Hi, Here is the complete yolo_nms function that i have been using. (mostly modified based on your suggestions) def yolo_nms(outputs, anchors, masks, classes):

b, c, t = [], [], []

for o in outputs:
    b.append(tf.reshape(o[0], (tf.shape(o[0])[0], -1, tf.shape(o[0])[-1])))
    c.append(tf.reshape(o[1], (tf.shape(o[1])[0], -1, tf.shape(o[1])[-1])))
    t.append(tf.reshape(o[2], (tf.shape(o[2])[0], -1, tf.shape(o[2])[-1])))

bbox = tf.concat(b, axis=1)
confidence = tf.concat(c, axis=1)
class_probs = tf.concat(t, axis=1)
scores = confidence * class_probs

'''
boxes, scores, classes, valid_detections = tf.image.combined_non_max_suppression(
        boxes = tf.reshape(bbox, (tf.shape(bbox)[0], -1, 1, 4)),
        scores = tf.reshape(scores, (tf.shape(scores)[0], -1, tf.shape(scores)[-1])),
        max_output_size_per_class=100, max_total_size=100, iou_threshold=0.5, score_threshold=0.5)

return boxes, scores, classes, valid_detections'''

sscores = tf.squeeze(scores, axis=0)
dscores = tf.reduce_max(sscores, [1]) #this will have shape(10647)
bbox = tf.reshape(bbox, (-1, 4))
#classes = tf.argmax(classes, 1)
classes = tf.reshape(classes,[1])

selected_indices, selected_scores = tf.image.non_max_suppression_with_scores(boxes=bbox, scores=dscores, max_output_size=100, iou_threshold=0.5, score_threshold=0.5, soft_nms_sigma=0.5)

num_valid_nms_boxes = tf.shape(selected_indices)[0]
selected_indices = tf.concat([selected_indices, tf.zeros(100-num_valid_nms_boxes, tf.int32)], -1)
selected_scores = tf.concat([selected_scores, tf.zeros(100-num_valid_nms_boxes, tf.float32)], -1)
classes = tf.concat([classes, tf.ones(100 - 1, tf.int32)], 0)

boxes = tf.gather(bbox, selected_indices)
boxes = tf.expand_dims(boxes, axis=0)
scores = selected_scores
scores = tf.expand_dims(scores, axis=0)
classes = tf.expand_dims(classes, axis=0)
valid_detections = num_valid_nms_boxes
valid_detections = tf.expand_dims(valid_detections, axis=0)

return boxes, scores, classes, valid_detections

Hi @VeeranjaneyuluToka Please comment these two lines from your code:

classes = tf.argmax(classes, 1)

classes = tf.reshape(classes,[1])

And instead of above mentioned two lines change to: classes = tf.argmax(sscores,1)#this will have shape(10647)

After that comment that line also: classes = tf.concat([classes, tf.ones(100 - 1, tf.int32)], 0)

And before that line: classes = tf.expand_dims(classes, axis=0) Add this: classes = tf.gather(classes,selected_indices)

Please make above mentioned changes and most probably it has to work

VeeranjaneyuluToka commented 3 years ago

@rajan780 , that did the trick. Thanks!

rajan780 commented 3 years ago

@rajan780 , that did the trick. Thanks!

it's my pleasure @VeeranjaneyuluToka

VeeranjaneyuluToka commented 3 years ago

Am wondering in case if you have any experience in converting from savedmodel -> onnx -> tensorrt. I could convert from savedmodel format to onnx but not able to convert from onnx to tensorrt.

rajan780 commented 3 years ago

Am wondering in case if you have any experience in converting from savedmodel -> onnx -> tensorrt. I could convert from savedmodel format to onnx but not able to convert from onnx to tensorrt.

I converted my model into tensorrt by using below mentioned repo: https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/DS2 repo

VeeranjaneyuluToka commented 3 years ago

Hi Again, the above repository is basically running the darknet models. I have tensorflow models converted to onnx and need to convert to tensorrt. Have you converted your tensorflow model to tensorrt using the above repo?

rajan780 commented 3 years ago

Hi Again, the above repository is basically running the darknet models. I have tensorflow models converted to onnx and need to convert to tensorrt. Have you converted your tensorflow model to tensorrt using the above repo?

Hi @VeeranjaneyuluToka No,i didn't convert from onnx to tensorrt. i convert from darknet to tensorrt only.

VeeranjaneyuluToka commented 3 years ago

I have trained a custom yolov3 model using tensorflow 2 which means that the model is in the format of savedmodel. Originally, tf.image.combined_non_max_suppression is being used in inference which is not supported by onnx, so getting error while converting from savedmodel to onnx.

This issue is solved by replacing tf.image.combined_non_max_suppression with tf.image.non_max_suppresion_with_scores (check the above discussion on this). I am able to convert the model from savedmodel -> onnx with the above changes.

And results of savedmodel and onnx model are identical. however i am trying to convert from onnx to trt and getting below message

Input filename: /tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/20210209-214418_convert/model_15.onnx ONNX IR version: 0.0.7 Opset version: 13 Producer name: tf2onnx Producer version: 1.8.2 Domain:
Model version: 0 Doc string:

[02/16/2021-09:53:34] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. terminate called after throwing an instance of 'std::out_of_range' what(): Attribute not found: axes Aborted (core dumped)

Anybody has faced this issue and found any workaround?

rajan780 commented 3 years ago

I have trained a custom yolov3 model using tensorflow 2 which means that the model is in the format of savedmodel. Originally, tf.image.combined_non_max_suppression is being used in inference which is not supported by onnx, so getting error while converting from savedmodel to onnx.

This issue is solved by replacing tf.image.combined_non_max_suppression with tf.image.non_max_suppresion_with_scores (check the above discussion on this). I am able to convert the model from savedmodel -> onnx with the above changes.

And results of savedmodel and onnx model are identical. however i am trying to convert from onnx to trt and getting below message

Input filename: /tf_git_hubs/yolo_tf/yolov3-tf2-master/model_outputs/20210209-214418_convert/model_15.onnx

ONNX IR version: 0.0.7 Opset version: 13 Producer name: tf2onnx Producer version: 1.8.2 Domain: Model version: 0 Doc string: [02/16/2021-09:53:34] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. terminate called after throwing an instance of 'std::out_of_range' what(): Attribute not found: axes Aborted (core dumped)

Anybody has faced this issue and found any workaround?

@VeeranjaneyuluToka Please check that link: https://github.com/onnx/tensorflow-onnx/issues/883#issuecomment-614561227 may be it will help you.