Closed owengregson closed 1 year ago
Hello,
Still need some sort of resolution for this so I am posting another comment to bump it.
If you seriously read my response to the issue you posted the other day and tried to address it seriously, there is nothing more I can explain. Good luck.
I am not a coding teacher. Duplicate of: https://github.com/PINTO0309/PINTO_model_zoo/issues/344
Issue Type
Bug
OS
Mac OS
OS architecture
x86_64, armv7, armv6
Programming Language
Other
Framework
CoreML
Model name and Weights/Checkpoints URL
https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/287_Topformer/resources.tar.gz
"model_coreml_float32.mlmodel"
Description
Our testing uses the Swift programming language, as CoreML models are designed to be integrated into Swift apps.
The Issue
The CoreML model should take an
Image
type as input but instead takes anMLMultiArray
. Because of this, the model is effectively unusable in Swift, and any attempts to use it usually result in the error listed in the Log Output below. The error basically means that the model is unable to accept anImage
as input, even though it is designed to process images.What we tried
We attempted to change the input type ourselves using Python's
coremltools
, which seemed to have succeeded, but after importing the model into XCode... The editor was unable to build the project without encountering a build error. The error relates to something about the kernel size being incorrect for the reshape operation.We also spent a long time trying to convert an ONNX version to CoreML ourselves, but due to the recent drop of ONNX -> CoreML support, we merely encountered issue after issue and ultimately failed.
We also tried a myriad of other methods to convert Tensorflow, PyTorch, and other model types to CoreML, but it was more of the same and we just encountered numerous issues that eventually made us give up.
If you wish to test the model in Swift, you can use the code I provided at the bottom of this issue to import the model, load it, then make a request to process something.
The Solution
However, all hope is not lost. I believe this issue should be simple to fix; whatever conversion process was originally used to convert the model into CoreML should be altered so that the model can properly accept
Image
as an input.In addition, you could add (or replace the old model) file to the .tar.gz in the TopFormer folder in this repo so that others trying to use the CoreML version in their Swift app will not encounter runtime errors or build errors.
TLDR
The model needs to accept
Image
but acceptsMLMultiArray
instead. It needs to be re-converted appropriately.If you'd like any more information about my process, what we tried, or something else, please let me know.
Relevant Log Output
URL or source code for simple inference testing code