Can't convert to CoreML

[BadrinathS]

I am trying to convert coremel so I can implement this on swift. I am trying two approaches,

1. Using onnx to coreml
2. Using pth to coreml

However both the approaches giving error, below is the code I used

1. ONNX to CoreML

onnx_path = "./onnx_mobilenetv2_hd.onnx" ct.converters.onnx.convert(onnx_path, minimum_ios_deployment_target='13')

Error given on line ct.converters.onnx.convert NotImplementedError: Unsupported ONNX ops of type: ScatterElements,If,Range

2. Pth to CoreML

path = './torchscript_mobilenetv2_fp32.pth' saved_name = './mobilenetv2.mlmodel'

input_shape = ct.Shape(shape=(3, ct.RangeDim(1, 360), ct.RangeDim(1, 360, default=340), ct.RangeDim(1, 360, default=340) )) model_input_1 = ct.ImageType(name='src', shape=input_shape,scale=1/127.5, bias=[-1, -1, -1]) model_input_2 = ct.ImageType(name='bgr', shape=input_shape,scale=1/127.5, bias=[-1, -1, -1]) mlmodel = ct.convert(path, source='pytorch', inputs=[model_input_1, model_input_2],input_names=["src", "bgr"], image_input_names=["src", "bgr"]) mlmodel.save(saved_name)

Error given on line ct.convert RuntimeError: temporary: the only valid use of a module is looking up an attribute but found = prim::SetAttr[name="num_batches_tracked"](%5640, %5647)

Thanks. Looking forward to suggestions.

[PeterL1n]

1. You can get rid of the ScatterElements op by re-export ONNX yourself using the export_onnx.py script with argument --model-refine-patch-replace-method scatter_nd instead. This replace ScatterElements with ScatterND

You can also get rid of the if op by going into model/refiner.py and change line 96 from idx = torch.nonzero(ref.squeeze(1)) to idx = torch.nonzero(ref[:, 0])

Not sure where the Range op is from.

2. Maybe you didn't call model.eval() before?

[PeterL1n]

I have also answered some previous CoreML questions here https://github.com/PeterL1n/BackgroundMattingV2/issues/94

[BadrinathS]

Thanks for replying.

In 2, I am providing path to pth file to ct.convert function. Where should I do model.eval() ? Should I load the model first and then pass it to ct.convert?

[BadrinathS]

For 2.

if args.model_type == 'mattingbase': model = MattingBase(args.model_backbone) if args.model_type == 'mattingrefine': model = MattingRefine( backbone=args.model_backbone, backbone_scale=args.model_backbone_scale, refine_mode=args.model_refine_mode, refine_sample_pixels=args.model_refine_sample_pixels, refine_threshold=args.model_refine_threshold, refine_kernel_size=args.model_refine_kernel_size, refine_patch_crop_method=args.model_refine_patch_crop_method, refine_patch_replace_method=args.model_refine_patch_replace_method)

model.load_state_dict(torch.load(path_pth, map_location=args.device), strict=False) model.eval() mlmodel = ct.convert(model, source='pytorch', inputs=[model_input_1, model_input_2],input_names=["src", "bgr"], image_input_names=["src", "bgr"])

2 cases arises from here,

Case 1. When I use path_pth for Pytorch weights file then error shows up at ct.convert line

TypeError: @model must either be a PyTorch .pt or .pth file or a TorchScript object, received: <class 'model.model.MattingBase'>

This means I have to provide path to the weights instead of loading the weights and providing the model. Now when I provide Pytorch weight file path to ct.convert it shows error on line ct.convert

RuntimeError: [enforce fail at inline_container.cc:222] . file not found: archive/constants.pkl

Reading more into the second error shows that maybe the Pytorch path should be torch script path because it mentions return _torch.jit.load(filename).

Case 2. When I provide torch script path directly to ct.convert it shows the error that I originally mentioned when I posted this question

RuntimeError: temporary: the only valid use of a module is looking up an attribute but found = prim::SetAttr[name="num_batches_tracked"](%5640, %5647)

And when I use torch script file to load the model and pass the model to ct.convert after model.eval() it shows error on line model.load_state_dict(torch.load(path_script, map_location=args.device), strict=False) as

torch.nn.modules.module.ModuleAttributeError: 'RecursiveScriptModule' object has no attribute 'copy'

So what should I do from here if the goal is to convert to coreml from pth file?

[PeterL1n]

If you are still stuck here,

You should not do case 1. If they ask for torchscript, then you need to give it torchscript. Approach 2 is the way to go.

The error in case 2 is because we exported torchscript as torch.jit.script(model). Which exports the entire model logics including some of the logic in batchnorm layers that is only used for training. To avoid this. You can export torchscript again using torch.jit.trace(model).