Conversion process Readme File and Sample Swift App for YOLO v5

[NaeemKhan333]

@pytholic Thank you very much for such nice work. I just want to get guidance about the conversion/export process of the yolov5 model. Secondly, if you have any sample app for a converted model of yolov5s which can be used for integration. Thank you

[pytholic]

@pytholic Thank you very much for such nice work. I just want to get guidance about the conversion/export process of the yolov5 model. Secondly, if you have any sample app for a converted model of yolov5s which can be used for integration. Thank you

@NaeemKhan333 How about this notebook for conversion process?

You can check a sample app here. It is for detection breast region of patient for medical purposes, or you can also check out these two links which helped me during my work:

Ma-Dan Hollance

Let me know if it helps :)

[NaeemKhan333]

@pytholic which version of yolov5, you have used for training. I used the latest version of yolov5 for training my custom model. So when I convert it to the coreML model, it does not have a preview tab (as your model has in XCode). Can you give me guidance about the training process and version? Secondly what I do in conversion/export so my model work in the Xcode and iOS device. Thanks

[NaeemKhan333]

I have to download the yolov5s (v4 and downloaded its pre-trained weights) and yolov5-core ml tools. But when I run

$ poetry install

I am getting following error

  Directory /home/user/Downloads/yolov5-4.0/yolov5 does not seem to be a Python package

  at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/packages/directory_dependency.py:53 in __init__
       49│             self._full_path / "pyproject.toml"
       50│         ).is_poetry_project()
       51│ 
       52│         if not setup.exists() and not self._supports_poetry:
    →  53│             raise ValueError(
       54│                 "Directory {} does not seem to be a Python package".format(
       55│                     self._full_path
       56│                 )
       57│             )

[pytholic]

I have to download the yolov5s (v4 and downloaded its pre-trained weights) and yolov5-core ml tools. But when I run

$ poetry install

I am getting following error

  Directory /home/user/Downloads/yolov5-4.0/yolov5 does not seem to be a Python package

  at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/packages/directory_dependency.py:53 in __init__
       49│             self._full_path / "pyproject.toml"
       50│         ).is_poetry_project()
       51│ 
       52│         if not setup.exists() and not self._supports_poetry:
    →  53│             raise ValueError(
       54│                 "Directory {} does not seem to be a Python package".format(
       55│                     self._full_path
       56│                 )
       57│             )

You also need to clone (or download and extract) the Yolov5 v4.0 repository and place it in the same folder as yolov5-coreml-tools.

[pytholic]

@pytholic which version of yolov5, you have used for training. I used the latest version of yolov5 for training my custom model. So when I convert it to the coreML model, it does not have a preview tab (as your model has in XCode). Can you give me guidance about the training process and version? Secondly what I do in conversion/export so my model work in the Xcode and iOS device. Thanks

Can you tell me what you mean by preview tab? Maybe share a screenshot?

I used v6.0 for training. During conversion with poetry, you also need a copy of original repository in your folder (source code). I download that but due to some reason, v6.0 gave me error. So I downloaded source code of v4.0 and the nit worked.

To summarize, my model is v6.0 and source code (required during poetry export) is v4.0.

Coming to your last question, one you have the .coreml model, you can load it inside Xcode and use it in your applciations.

[NaeemKhan333]

@pytholic I put the yolov5 (v4) code and its pretrained yolov5s weight(v4) downloaded from the following link

https://github.com/ultralytics/yolov5/releases/tag/v4.0

and

for conversion, I use this tool

https://github.com/dbsystel/yolov5-coreml-tools

My folders look like this

My pyproject.toml

# Copyright (C) 2021 DB Systel GmbH.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

[tool.poetry]
name = "coreml-tools"
version = "0.2.1"
description = "Different scripts to use YOLOv5 with coreml"
authors = ["Leon De Andrade"]
license = "Apache 2.0"
readme = "README.md"
packages = [
    { include = "coreml_export", from = "src"}
]

[tool.poetry.dependencies]
# Common
python = ">=3.7, <3.9"

# coreml export 
coremltools = "^4.1"
yolov5 = { path = "../yolov5", develop = true }

# Workaround for Windows
torch = [
    {version = "=1.7.0", markers="sys_platform != 'win32'"},
    # {url = "https://download.pytorch.org/whl/cu102/torch-1.6.0-cp38-cp38-win_amd64.whl", python = "~3.8", markers = "sys_platform == 'win32'"}
]
torchvision = [
    {version = "=0.8.0",  markers = "sys_platform != 'win32'"},
    # {url = "https://download.pytorch.org/whl/cu102/torchvision-0.7.0-cp38-cp38-win_amd64.whl", python = "~3.8", markers = "sys_platform == 'win32'"}
]

[tool.poetry.scripts]
coreml-export = "coreml_export.main:main"
coreml-test = "coreml_export.test:main"
coreml-metrics = "coreml_metrics.main:main"

[build-system]
requires = ["poetry_core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[NaeemKhan333]

How do you install poetry ? Which version of poetry you are using while in conversion?

[pytholic]

you are missing yolov5 = { path = "../yolov5", develop = true } under [tool.poetry.dependencies]. I have attached my file.

[pytholic]

@NaeemKhan333 Your folder structure looks good. No issue there. I install poetry with conda link

[NaeemKhan333]

Ok Thank you , I am trying it and reply you back in few minutes

[NaeemKhan333]

@pytholic I have resolved the issue by using your clone of yolov5-coreml-tools in your repository

https://github.com/pytholic/Yolov5Export/tree/main/poetry_yolov5/yolov5-coreml-tools

and using the yolov5 code in your repository

https://github.com/pytholic/Yolov5Export/tree/main/poetry_yolov5/yolov5

I converted the pre-trained weights of yolov5s(v4) into core ML successfully. But when I try to convert my own model yolov5s (trained on single class with yolov5(v6)) . Then I am getting this error

poetry run coreml-export --model-input-path ../yolov5/yolov5s.pt 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/user/Documents/Yolov5Export-main (1)/My-Conversion/yolov5-coreml-tools/src/coreml_export/main.py", line 312, in main
    model = torch.load(opt.model_input_path, map_location=torch.device('cpu'))[
  File "/home/user/.cache/pypoetry/virtualenvs/coreml-tools-kcCJAqXp-py3.7/lib/python3.7/site-packages/torch/serialization.py", line 594, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/user/.cache/pypoetry/virtualenvs/coreml-tools-kcCJAqXp-py3.7/lib/python3.7/site-packages/torch/serialization.py", line 853, in _load
    result = unpickler.load()
AttributeError: Can't get attribute 'SPPF' on <module 'models.common' from '/home/user/Documents/Yolov5Export-main (1)/My-Conversion/yolov5/m

I run the detect.py (yolov5-v4) with my model (trained yolov5-v6) and It also gives the same error. I think newly trained models are not supported on an old version of code?

https://github.com/ultralytics/yolov5/issues/5175#issuecomment-976799374

python detect.py 
/home/user/anaconda3/lib/python3.7/site-packages/pandas/compat/_optional.py:138: UserWarning: Pandas requires version '2.7.0' or newer of 'numexpr' (version '2.6.9' currently installed).
  warnings.warn(msg, UserWarning)
Namespace(agnostic_nms=False, augment=False, classes=None, conf_thres=0.25, device='', exist_ok=False, img_size=640, iou_thres=0.45, name='exp', project='runs/detect', save_conf=False, save_txt=False, source='data/images', update=False, view_img=False, weights='yolov5s.pt')
Using torch 1.7.1+cpu CPU

Traceback (most recent call last):
  File "detect.py", line 172, in <module>
    detect()
  File "detect.py", line 33, in detect
    model = attempt_load(weights, map_location=device)  # load FP32 model
  File "/home/user/Documents/Yolov5Export-main (1)/My-Conversion/yolov5/models/experimental.py", line 118, in attempt_load
    model.append(torch.load(w, map_location=map_location)['model'].float().fuse().eval())  # load FP32 model
  File "/home/user/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 594, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/user/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 853, in _load
    result = unpickler.load()
AttributeError: Can't get attribute 'SPPF' on <module 'models.common' from '/home/user/Documents/Yolov5Export-main (1)/My-Conversion/yolov5/m

Can you help me out? Thanks

[NaeemKhan333]

@pytholic Thank you very much, I am able to convert the yolov5s model into coreML format. By doing the following things

1- Clone of yolov5-coreml-tools of your repository

https://github.com/pytholic/Yolov5Export/tree/main/poetry_yolov5/yolov5-coreml-tools

2- Use the yolov5 code of your repository

https://github.com/pytholic/Yolov5Export/tree/main/poetry_yolov5/yolov5

3- Train the custom model on Yolov5 version 4

https://github.com/ultralytics/yolov5/releases/tag/v4.0

4- Place both folders in the same folder and follow this repository's readme

https://github.com/dbsystel/yolov5-coreml-tools

It is working fine in Xcode.

[NaeemKhan333]

@pytholic my question is related to the conversion of the model when I train the model on yolov5 (version-6). It is giving me the above error which I mentioned. Can you guide me, how you converted the model of yolov5 (v6) using the yolov5(v4) code? Can you help me out? Thanks

[NaeemKhan333]

@pytholic Secondly can we convert the yolov5m.pt into coreML . Do we need a different way to convert others' yolov5 weights? Can you guide me about it? Thanks

[pytholic]

@pytholic Secondly can we convert the yolov5m.pt into coreML . Do we need a different way to convert others' yolov5 weights? Can you guide me about it? Thanks

I think we can convert any .pt model to coreml by same method. Refer to this notebook in my repos.

Actually there are two ways to convert. First one is in the official Yolov5 repo, but the issue with this one is that Non-max suppression is not integrated in the model. Rather it is implemented separately in the detect.py script. Second method is based on dbsystel repo, and it includes non-max suppression in the model. This is the method that you and I both used.

Regarding your other concern, for me, when I used yolov5(4.0) and my model from yolov5(6.0), the conversion worked fine. The issue was that when I used yolov5(6.0) and my model from yolov5(6.0, I got the following error.

RuntimeError: The expanded size of the tensor (1) must match the existing size (80) at non-singleton dimension 3.  Target sizes: [1, 3, 1, 1, 2].  Tensor sizes: [3, 80, 80, 2]

Your issue seems to be mentioned here. Seems like there have been some updates in the code and trying to use new models with the old code can cause this issue. Solution seems to be updating the code, but for us it is not possible because we have to use yolov5(4.0 repository during the conversion. If we use newer version then dbsystel repo gives us the error which I mentioned above.

I think only option for now would be to retrain your model using little older version of the code. See this reply. Other option can be to use official export.py script and use official detect.py, but then the non-max suppression is not included in the mode. If you create a standalone application with coreml model, then you might have to implement nms manually which can be hard.

I hope it helps!

[NaeemKhan333]

@pytholic thanks for the reply. I trained my model on yolov5 (v4), and convert .pt weight into coreML successfully and it is working fine.Thanks

[pytholic]

@pytholic thanks for the reply. I trained my model on yolov5 (v4), and convert .pt weight into coreML successfully and it is working fine.Thanks

@NaeemKhan333 That's great, cheers :)

[NaeemKhan333]

@pytholic sorry for bothering again, Have you converted the yolov3 model into CoreML format. Can you guide me about it. Thanks

[pytholic]

@NaeemKhan333 Hi. Sorry but I haven't tried the YoloV3 model. Maybe try to look here in the official repo if they have some info on it.