[Core ML] Rank-0 Input

[YifanShenSZ]

When we try to partition certain models on to Core ML delegation, we were hit by

ValueError: Rank-0 (input None) is unsupported

This is because Core ML does not support rank-0 (scalar) input, yet ExecuTorch tries to send a graph with scalar input to Core ML backend

We will need to wrap those scalars into rank-1 shape-1 tensors before sending to Core ML backend

[cccclai]

From top of my brain, we can have a pass to convert all scalar to tensor before lower it to CoremL

[cymbalrush]

@YifanShenSZ is this still an issue?

[YifanShenSZ]

No longer urgent.

Concretely, the rank-0 "inputs" we encountered were constants, that ExecuTorch decided to keep and feed to backend during runtime. By letting Core ML backend own those constants (i.e. feed at AOT), we no longer see them as inputs.

[cccclai]

No longer urgent.

Concretely, the rank-0 "inputs" we encountered were constants, that ExecuTorch decided to keep and feed to backend during runtime. By letting Core ML backend own those constants (i.e. feed at AOT), we no longer see them as inputs.

Does it mean if there is real rank-0 input, the issue might still be there? If it is, it will be great to share an example, and I will change the tag from coreml to exir, as it sounds like a pass we can provide.

[YifanShenSZ]

Yes, here is a minimum reproduce based on executorch 0.2.0

import torch
import executorch.exir

import coremltools as ct

class Model(torch.nn.Module):
    def forward(self, x, y):
        return x + y

model = Model()
model.eval()

example_inputs = (torch.tensor(1.0), torch.tensor(2.0))
exported_program_manager_aten = torch.export.export(model, example_inputs)
exported_program_manager_edge = executorch.exir.to_edge(exported_program_manager_aten)
print(exported_program_manager_edge.exported_program())

coreml_model = ct.convert(exported_program_manager_edge.exported_program())

The exported program has scalar inputs

ExportedProgram:
    class GraphModule(torch.nn.Module):
        def forward(self, arg0_1: "f32[]", arg1_1: "f32[]"):
            # File: /Volumes/Models/LLM/Framework/CoreMLTools-Dev_ExecuTorch-0.2/Debug/rank-0_input/rank-0_input.py:9 in forward, code: return x + y
            aten_add_tensor: "f32[]" = executorch_exir_dialects_edge__ops_aten_add_Tensor(arg0_1, arg1_1);  arg0_1 = arg1_1 = None
            return (aten_add_tensor,)

and CoreML conversion would fail

ValueError: Rank-0 (input None) is unsupported