search

ultralytics / ultralytics

NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
28.53k stars 5.67k forks source link

export RTDETR as coreML #16343

Open keisan1231 opened 1 day ago

keisan1231 commented 1 day ago

Search before asking

Question

How can I use RTDETR coreML??

Additional

I am trying to use RTDETR in an iOS application. Following this comment, I attempted to export the model in CoreML format, but I encountered the following error:

ValueError: In op, of type linear, named out_w, the named input `weight` must have the same data type as the named input `x`. However, weight has dtype fp32 whereas x has dtype int32.

code

from ultralytics import RTDETR

# Load your model
model = RTDETR('rtdetr-l.pt')

# Ensure model and inputs are in the same data type (e.g., float32)
model = model.float()

# Now try exporting again
model.export(format="coreml")
All detail log ``` [W NNPACK.cpp:64] Could not initialize NNPACK! Reason: Unsupported hardware. rt-detr-l summary: 494 layers, 32148140 parameters, 0 gradients, 103.8 GFLOPs PyTorch: starting from 'rtdetr-l.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 300, 84) (63.4 MB) scikit-learn version 1.5.1 is not supported. Minimum required version: 0.17. Maximum required version: 1.1.2. Disabling scikit-learn conversion API. Torch version 2.2.2 has not been tested with coremltools. You may run into unexpected errors. Torch 2.2.0 is the most recent version that has been tested. CoreML: starting export with coremltools 7.2... Converting PyTorch Frontend ==> MIL Ops: 0%| | 0/2329 [00:00 MIL Ops: 25%|█████████▊ | 574/2329 [00:00<00:00, 5716.50 ops/s]Saving value type of float64 into a builtin type of fp32, might lose precision! ERROR - converting 'matmul' op (located at: '11'): Converting PyTorch Frontend ==> MIL Ops: 25%|██████████ | 584/2329 [00:00<00:00, 5540.22 ops/s] CoreML: export failure ❌ 20.0s: In op, of type linear, named out_w, the named input weight must have the same data type as the named input x. However, weight has dtype fp32 whereas x has dtype int32. Traceback (most recent call last): File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/ml_xcode_create.py", line 10, in model.export(format="coreml") File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/ultralytics/engine/model.py", line 591, in export return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/ultralytics/engine/exporter.py", line 310, in __call__ f[4], _ = self.export_coreml() File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/ultralytics/engine/exporter.py", line 142, in outer_func raise e File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/ultralytics/engine/exporter.py", line 137, in outer_func f, model = inner_func(*args, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/ultralytics/engine/exporter.py", line 633, in export_coreml ct_model = ct.convert( File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 581, in convert mlmodel = mil_convert( File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert proto, mil_program = mil_convert_to_proto( File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 288, in mil_convert_to_proto prog = frontend_converter(model, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 108, in __call__ return load(*args, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 82, in load return _perform_torch_convert(converter, debug) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 116, in _perform_torch_convert prog = converter.convert() File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 581, in convert convert_nodes(self.context, self.graph) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 86, in convert_nodes raise e # re-raise exception File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 81, in convert_nodes convert_single_node(context, node) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 134, in convert_single_node add_op(context, node) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 879, in matmul res = mb.linear(x=linear_x, weight=transposed_weight, name=node.name) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 182, in add_op return cls._add_op(op_cls_to_add, **kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/mil/builder.py", line 182, in _add_op new_op = op_cls(**kwargs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/mil/operation.py", line 191, in __init__ self._validate_and_set_inputs(input_kv) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/mil/operation.py", line 504, in _validate_and_set_inputs self.input_spec.validate_inputs(self.name, self.op_type, input_kvs) File "/Users/yamaguchikeiichi/Downloads/smart-judge/ball_detect_coreml_create/testEnv/lib/python3.10/site-packages/coremltools/converters/mil/mil/input_type.py", line 137, in validate_inputs raise ValueError(msg) ValueError: In op, of type linear, named out_w, the named input weight must have the same data type as the named input x. However, weight has dtype fp32 whereas x has dtype int32. ```
UltralyticsAssistant commented 1 day ago

👋 Hello @keisan1231, thank you for reaching out and for your interest in using Ultralytics 🚀! This is an automated response, and an Ultralytics engineer will assist you soon.

We recommend checking the Documentation where you'll find numerous Python and CLI examples that might clarify your doubts.

For your CoreML export attempts, ensure all inputs and model parameters are of the same dtype. If this is a 🐛 Bug Report, please provide a minimum reproducible example to assist us in debugging the issue.

If you're diving into custom training or have other ❓ Questions, kindly include detailed info like dataset samples and training logs. Also, review our Tips for Best Training Results to ensure optimal outcomes.

Stay connected with our community! For real-time interactions, join us on Discord 🎧. Prefer detailed discussions? Explore our Discourse or engage on our Subreddit.

Upgrade

First, verify your issue against the latest version of the package. Upgrade using:

pip install -U ultralytics

Ensure all requirements are updated in a Python>=3.8 environment and PyTorch>=1.8.

Environments

Check these verified environments with all required dependencies (including CUDA/CUDNN) for running YOLOv8:

Status

Ultralytics CI

When the badge is green, all Ultralytics CI tests are passing. CI tests ensure correct operation across all YOLOv8 Modes and Tasks on various platforms.

We're here to help and make your experience smooth 🌟!