Topformer coreml wrong input type

[Peter-Zhao-751]

Issue Type

Bug

OS

Mac OS

OS architecture

x86_64

Programming Language

Python, Other

Framework

CoreML

Model name and Weights/Checkpoints URL

model_coreml_float32.mlmodel

https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/287_Topformer/resources.tar.gz

Description

the model's input type is a multiarray instead of an image. This causes the model to not function when used in swift with the error of "Error Domain=com.apple.vis Code=15 "The model does not have a valid input feature of type image" UserInfo={NSLocalizedDescription=The model does not have a valid input feature of type image}." I also tried to change the model's input with coremltools, but ended up with a build error. I believe the problem is easily solvable with the correct conversion process.

Relevant Log Output

Error Domain=com.apple.vis Code=15 "The model does not have a valid input feature of type image" UserInfo={NSLocalizedDescription=The model does not have a valid input feature of type image}

URL or source code for simple inference testing code

guard let segmentationModelURL = Bundle.main.url(forResource: "model_coreml_float32", withExtension: "mlmodelc") else { print("missing segmentation model url") return } do { let segmentationModel = try VNCoreMLModel(for: MLModel(contentsOf: segmentationModelURL)); segmentationRequest = VNCoreMLRequest(model: segmentationModel); print("thing")} catch{ print(error)}

func makeMLRequest(sampleBuffer: CMSampleBuffer, request: VNRequest){
    let requestHandler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer)
    do {
        try requestHandler.perform([request])
    } catch {
        print(error)
    }
}

[PINTO0309]

Having never used Swift, I have no idea what the error message is intended to mean. Why don't you try reconverting it yourself?

onnx2tf \
-i topformer_base_512x512_argmax.onnx \
-osd \
-kat first_input \
-coion

import coremltools as ct

FOLDER_PATH = 'saved_model'

model = ct.convert(
    model=FOLDER_PATH,
    source='tensorflow',
)
model.save(f'{FOLDER_PATH}/model.mlmodel')

[Peter-Zhao-751]

Thanks for your help, but I don't know how to convert a tensorflow or onnx to coreml. Would you mind sending me a pretrained pytorch model? Thanks.

[PINTO0309]

I have no idea what you are talking about. Have you not unzipped the download file you mentioned in your first comment and looked at the contents?

I don't like messy comments.