ultralytics / yolov5

YOLOv5 πŸš€ in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
50.26k stars 16.23k forks source link

Got error when export TFlite with NMS #6799

Closed chinaandylee closed 2 years ago

chinaandylee commented 2 years ago

Search before asking

Question

When I export tflite model with opt --nms python export.py --weights yolov5s.pt --include tflite --nms I got the following error: ..... tf.expand_dims (TFOpLambda) (1, 25200, 1, 4) 0 tf.concat[0][0]


tf.math.multiply (TFOpLambda) (1, 25200, 2) 0 tf.operators.getitem_1[0][0] tf.operators.getitem_2[0][0]


tf.image.combined_non_max_suppr CombinedNonMaxSuppre 0 tf.expand_dims[0][0] tf.math.multiply[0][0]

Total params: 7,015,519 Trainable params: 0 Non-trainable params: 7,015,519


2022-02-27 14:40:41.858021: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider a voiding using them. Found untraced functions such as tf_conv_2_layer_call_fn, tf_conv_2_layer_call_and_return_conditional_losses, tf_conv_3_layer_call_fn, tf_conv_3_layer_call_and _return_conditional_losses, tf_conv_4_layer_call_fn while saving (showing 5 of 520). These functions will not be directly callable after loading. Found untraced functions such as tf_conv_2_layer_call_fn, tf_conv_2_layer_call_and_return_conditional_losses, tf_conv_3_layer_call_fn, tf_conv_3_layer_call_and _return_conditional_losses, tf_conv_4_layer_call_fn while saving (showing 5 of 520). These functions will not be directly callable after loading. Assets written to: last_saved_model\assets TensorFlow SavedModel: export success, saved as last_saved_model (238.6 MB)

TensorFlow Lite: starting export with tensorflow 2.4.0... Found untraced functions such as tf_conv_2_layer_call_fn, tf_conv_2_layer_call_and_return_conditional_losses, tf_conv_3_layer_call_fn, tf_conv_3_layer_call_and _return_conditional_losses, tf_conv_4_layer_call_fn while saving (showing 5 of 520). These functions will not be directly callable after loading. Found untraced functions such as tf_conv_2_layer_call_fn, tf_conv_2_layer_call_and_return_conditional_losses, tf_conv_3_layer_call_fn, tf_conv_3_layer_call_and _return_conditional_losses, tf_conv_4_layer_call_fn while saving (showing 5 of 520). These functions will not be directly callable after loading. Assets written to: C:\Users\kk\AppData\Local\Temp\tmp7pgu55vr\assets 2022-02-27 14:41:35.278574: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 2022-02-27 14:41:35.281165: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session 2022-02-27 14:41:35.283499: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set 2022-02-27 14:41:35.297874: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:928] Optimization results for grappler item: graph_to_optimize function_optimizer: function_optimizer did nothing. time = 0.001ms. function_optimizer: function_optimizer did nothing. time = 0ms.

2022-02-27 14:41:36.286075: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format. 2022-02-27 14:41:36.288583: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency. 2022-02-27 14:41:36.360661: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set loc(callsite("model/tf.image.combined_non_max_suppression/combined_non_max_suppression/CombinedNonMaxSuppression"("C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\s ite-packages\tensorflow\python\eager\def_function.py":1205:0) at callsite("C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\python\eager\def _function.py":1299:0 at callsite("C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\lite\python\lite.py":847:0 at callsite("export.py":324:0 at callsite("export.py":488:0 at callsite("C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\autograd\grad_mode.py":28:0 at callsite("export.py":5 39:0 at "export.py":544:0)))))))): error: 'tf.CombinedNonMaxSuppression' op is neither a custom op nor a flex op error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag): tf.CombinedNonMaxSuppression {clip_boxes = false, device = "", pad_per_class = false}

TensorFlow Lite: export failure: C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\python\eager\def_function.py:1205:0: error: 'tf.CombinedNo nMaxSuppression' op is neither a custom op nor a flex op C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\python\eager\def_function.py:1299:0: note: called from C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\lite\python\lite.py:847:0: note: called from export.py:324:0: note: called from export.py:488:0: note: called from C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\autograd\gradmode.py:28:0: note: called from export.py:539:0: note: called from export.py:544:0: note: called from **### :0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag): tf.CombinedNonMaxSuppression {clip_boxes = false, device = "", pad_perclass = false}**

Export complete (65.58s) Results saved to E:\python_resource\code_resource\pytorch\yolov5-master Detect: python detect.py --weights last_saved_model PyTorch Hub: model = torch.hub.load('ultralytics/yolov5', 'custom', 'last_saved_model') Validate: python val.py --weights last_saved_model Visualize: https://netron.app

Any help will be appreciated!

Additional

No response

github-actions[bot] commented 2 years ago

πŸ‘‹ Hello @chinaandylee, thank you for your interest in YOLOv5 πŸš€! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a πŸ› Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

CI CPU testing

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

leeflix commented 2 years ago

You should be able to fix this by changing line 323 in export.py from:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]

to:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

chinaandylee commented 2 years ago

According to your suggestion, I revised that line to converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS], but the problem remains. See the following for details. TensorFlow Lite: export failure: C:\Users\kk\Anaconda3\envs\pytorch_gpu\lib\site-packages\tensorflow\python\eager\def_function.py:1205:0: error: 'tf.CombinedNo nMaxSuppression' op is neither a custom op nor a flex op

leeflix commented 2 years ago

Can you please double check that you changed the correct line, because the line appears twice in the file.

image

chinaandylee commented 2 years ago

I have rechecked the revised lines. but the error still remain.

image

glenn-jocher commented 2 years ago

You should be able to fix this by changing line 323 in export.py from:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]

to:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

@leeflix should we make this a permanent change in the export code? Will this affect non-nms exports?

leeflix commented 2 years ago

@glenn-jocher CombinedNonMaxSuppression (--nms) and NonMaxSuppression (--agnostic-nms) are both Supported Select TensorFlow operators according to this. This suggests the changes I proposed. Not adding them seems to produce this issue so I would make this a permanent change in the export code. I made a PR #7281 that won't affect non-nms exports.

leeflix commented 2 years ago

@chinaandylee Mh, that is strange. Maybe make a fresh clone from my fork that tries to fix the issue and try again.

chinaandylee commented 2 years ago

With your fork, it works! Thanks! image

glenn-jocher commented 2 years ago

@chinaandylee @leeflix good news πŸ˜ƒ! Your original issue may now be fixed βœ… in PR #7281 by @leeflix. To receive this update:

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 πŸš€!

Euijune commented 2 years ago

@glenn-jocher

Hi, I understand that when I use export.py to change the .pt file to a .tflite file, if I give --nms option, the output comes in the form of nmsed_boxes (1,100), nmsed_scores=(1,100), nmsed_classes (1,100), valid_detections (1,100).

However, when I checked the output of tflite through detect.py, only nmsed_boxes existed, and scores, classes, and valid_detections were not printed.

The tflite model calculates the output properly, but did I make a mistake in the output? Or is it just the original nmsed_boxes?

Euijune commented 2 years ago

@glenn-jocher

The way I checked the output of the .tflite file is as follows. Line 122 of detect.py: pred = model (im, augment=augment, visualize=visualize) print(pred) and print(pred.shape) on the next line

result: pred shape: (1, 100, 4) <-- where is nmsed_score, nmsed_class, nmsed_valid detection??

AntoineChauviere commented 2 years ago

Hello! Did you find how to apply detect.py after exporting with --nms?

glenn-jocher commented 2 years ago

@AntoineChauviere detect.py works with all exports as long as they don't have NMS, as it applies torch NMS to all outputs.

AntoineChauviere commented 2 years ago

Ok thank you @glenn-jocher, how can I test an export model with nms then please?

glenn-jocher commented 2 years ago

@AntoineChauviere you'd have to write some custom code.

You can always test exports following the usage example after export, and of course we also run export benchmarks every 24 hours as part of our CI:

https://github.com/ultralytics/yolov5/actions/runs/3071491167

Screenshot 2022-09-17 at 12 36 08
AntoineChauviere commented 2 years ago

Ok thanks @glenn-jocher I'll try that then.

And last question, is it normal that I have 7 outputs? I thought I only had 4.

glenn-jocher commented 11 months ago

@AntoineChauviere Yes, for YOLOv5 the model typically produces 7 outputs: nmsed_boxes, nmsed_scores, nmsed_classes, valid_detections, output1, output2, output3.