search

nod-ai / SHARK-Turbine

Unified compiler/runtime for interfacing with PyTorch Dynamo.
Apache License 2.0
90 stars 45 forks source link

Quantized Grouped Convolution Support #678

Closed zjgarvey closed 3 months ago

zjgarvey commented 4 months ago

I am seeing 7 models failing due to quantized grouped convolutions being unsupported.

For example, 

ConvNeXt_vaiq_int8.default.torch-onnx.mlir:406:12: error: failed to legalize operation 'torch.aten.convolution' that was explicitly marked illegal
    %402 = torch.operator "onnx.Conv"(%389, %395, %401) {torch.onnx.dilations = [1 : si64, 1 : si64], torch.onnx.group = 128 : si64, torch.onnx.kernel_shape = [7 : si64, 7 : si64], torch.onnx.pads = [3 : si64, 3 : si64, 3 : si64, 3 : si64], torch.onnx.strides = [1 : si64, 1 : si64]} : (!torch.vtensor<[1,128,56,56],f32>, !torch.vtensor<[128,1,7,7],f32>, !torch.vtensor<[128],f32>) -> !torch.vtensor<[1,128,56,56],f32> 
           ^
ConvNeXt_vaiq_int8.default.torch-onnx.mlir:406:12: note: see current operation: %972 = "torch.aten.convolution"(%960, %962, %971, %454, %934, %454, %45, %453, %37) : (!torch.vtensor<[1,128,56,56],!torch.qint8>, !torch.vtensor<[128,1,7,7],!torch.qint8>, !torch.vtensor<[128],si32>, !torch.list<int>, !torch.list<int>, !torch.list<int>, !torch.bool, !torch.list<int>, !torch.int) -> !torch.vtensor<[1,128,56,56],si32>
zjgarvey commented 3 months ago

PR https://github.com/llvm/torch-mlir/pull/3341 merged so closing.