[Torch] A list of missing op conversion in need of help

[masahi]

Let's keep a list of missing conversion reported so far in one place here. The list will continue to grow as Torch has too many ops. cc @alexwong @jwfromm @pyjhzwh @jjohnson-arm

Implementing conversion should be straightforward following existing implementations. They are up for grab.

• [x] aten::reshape
• [x] aten::split
• [x] aten::max_pool1d
• [x] aten::embedding
• [x] prim::ImplicitTensorToNum
• [x] aten::layer_norm
• [x] aten::arange
• [x] aten::rsub
• [x] aten::gelu
• [x] aten::reciprocal
• [x] aten::repeat
• [x] aten::groupnorm

[megjoshi]

can we add these as well? 'prim::ImplicitTensorToNum', 'aten::layer_norm', 'aten::arange', 'aten::rsub', 'aten::gelu'

[masahi]

@megjoshi Updated. If possible, please show your original PyTorch models where these ops are generated from.

[megjoshi]

https://github.com/huggingface/transformers#quick-tour

this is on which i was trying to optimize using tvm

[NarendraPatwardhan]

Also aten::groupnorm as it is integral part of many new SOTA models and the alternative route pytorch->onnx->tvm introduces inefficient conversion by converting groupnorm to multiple instance norms instead.

[masahi]

@NarendraPatwardhan Group norm is supported in #5358.

[gopinath-r]

@masahi I am trying to use tvm for the following model [https://github.com/cogaplex-bts/bts/blob/master/pytorch/bts.py]

I am getting the following error while building the relay IR from pytorch scripted model TypeError: Don't know how to handle type <class 'torch.Tensor'>

After debugging i found that error is thrown at parsing the operation torch.repeat. and torch.repeat is mapped as aten::repeat in tvm/python/tvm/relay/frontend/pytorch.py

torch.repeat operates on tensor but while parsing torch.repeat it is unable to handle the data type

following is the complete error trace

` in 1 shape_list = [('input0', (1,3,1216,352))] 2 mod, params = relay.frontend.from_pytorch(scripted_model, ----> 3 shape_list)

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/relay/frontend/pytorch.py in from_pytorch(script_module, input_shapes, custom_convert_map) 2255 2256 ret = convert_operators(_get_operator_nodes(graph.nodes()), -> 2257 outputs, ret_name, convert_map, prelude) 2258 2259 mod["main"] = tvm.relay.Function(_analysis.free_vars(ret[0]), ret[0])

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/relay/frontend/pytorch.py in convert_operators(operators, outputs, ret_names, convert_map, prelude) 2169 relay_op = convert_map[operator] 2170 # print("relay_op",relay_op) -> 2171 relay_out = relay_op(inputs, _get_input_types(op_node)) 2172 2173 if isinstance(relay_out, tuple):

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/relay/frontend/pytorch.py in _impl(inputs, input_types) 322 # print("data",data) 323 # print("reps",reps) --> 324 return _op.transform.tile(data, reps=reps) 325 return _impl 326

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/relay/op/transform.py in tile(data, reps) 446 """ 447 --> 448 return _make.tile(data, reps) 449 450

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/_ffi/_ctypes/packed_func.py in call(self, *args) 209 """ 210 temp_args = [] --> 211 values, tcodes, num_args = _make_tvm_args(args, temp_args) 212 ret_val = TVMValue() 213 ret_tcode = ctypes.c_int()

~/Documents/Projects/Kyocera_depth_estimation/optimization/tvm/python/tvm/_ffi/_ctypes/packed_func.py in _make_tvm_args(args, temp_args) 175 else: 176 print(arg) --> 177 raise TypeError("Don't know how to handle type %s" % type(arg)) 178 return values, type_codes, num_args 179

TypeError: Don't know how to handle type <class 'torch.Tensor'>`

If you know what is the reason and how to resolve this it would be really helpful

Thanks in advance

[siju-samuel]

@gopinath-r can you please share the script to run from bts github or scripted_model? In repeat, can u please print both inputs, input_types and share the result.

[gopinath-r]

@siju-samuel thanks for the reply

These are the values of input and input_types in repeat input>>> bound method Kernel.raw_input of <ipykernel.ipkernel.IPythonKernel object at 0x7f1dc147c160> input_types>>> ['float', 'ListType'] data>>> tensor([[[0., 1., 2., 3., 4., 5., 6., 7.]]]) reps>>> [1, CallNode(Op(multiply), [Constant(152), Constant(8)], (nullptr), []), 44]

Use of torch.repeat can be found at line numbers 141, 144 of bts.py.

I have uploaded the script and necessary files to run bts in gdrive BTS_TVM

[siju-samuel]

@gopinath-r Currently tvm-pytorch frontend doesnt support taking each value of reps as another tensor or function. It is expecting as an simple constant and in this case we are getting a multiplication operator.

So as a workaround, you can modify your bts.py change your local_planar_guidance method typecast the scalar multiplication to int.

Modify the below two lines and do a simple typecast before multiplication.

    u = self.u.repeat(plane_eq.size(0), plane_eq.size(2) * int(self.upratio), plane_eq.size(3))
    v = self.v.repeat(plane_eq.size(0), plane_eq.size(2), plane_eq.size(3) * int(self.upratio))

to

    u = self.u.repeat(plane_eq.size(0), int(plane_eq.size(2) * int(self.upratio)), plane_eq.size(3))
    v = self.v.repeat(plane_eq.size(0), plane_eq.size(2), int(plane_eq.size(3) * int(self.upratio)))

Hope this will resolve your issue.

Please use https://discuss.tvm.ai/ from now onwards for discussing bugs and issues.

[woniuasd]

my pytorch code is "canvas[:, indices] = voxels"

The following operators are not implemented: ['aten::indexput']

[woniuasd]

@siju-samuel You mean i can use embedded instead of aten::indexput ？

[siju-samuel]

@woniuasd Can you share me the complete script or model where i can simulate this aten::index_put_ operator? Im not sure how to invoke this op.

[masahi]

@siju-samuel @woniuasd Since relay doesn't support in-place op, you cannot use that python idiom.

[tqchen]

@masahi shall we close the PR and open new ones since that all the lists are checked off?

[masahi]

@megjoshi All ops you requested have been implemented (big thanks to @siju-samuel ), please try again

[ShivaKothuru]

aten::reflectionpad2d, aten::elu please add these

[ShivaKothuru]

@masahi Please add aten::reflectionpad2d, aten::elu

[siju-samuel]

@ShivaKothuru #5624

[power0341]

NotImplementedError: The following operators are not implemented: ['aten::floor_divide']

can we have this guy 'aten::floor_divide' as well? it's required by shufflenetV2

[siju-samuel]

@power0341 Please apply the below patch and let me know.

diff --git a/python/tvm/relay/frontend/pytorch.py b/python/tvm/relay/frontend/pytorch.py
index f68affd82..f2e24a128 100644
--- a/python/tvm/relay/frontend/pytorch.py
+++ b/python/tvm/relay/frontend/pytorch.py
@@ -1760,6 +1760,7 @@ def _get_convert_map(prelude):
         "aten::arange"                          : _arange(),
         "aten::div"                             : _elemwise("divide"),
         "aten::div_"                            : _elemwise("divide"),
+        "aten::floor_divide"                    : _elemwise("floor_divide"),
         "aten::addcdiv"                         : _addcdiv(),
         "aten::addcmul"                         : _addcmul(),
         "aten::ones"                            : _ones(),

[power0341]

@siju-samuel thanks a lot, it works, good job

[SJTU-yys]

It seems that aten::norm is not supported. @siju-samuel could you please add this layer?

[shizukanaskytree]

Missing op aten::copy_:

error:

NotImplementedError: The following operators are not implemented: ['aten::copy_']

[wwwwcu]

NotImplementedError: The following operators are not implemented: ['prim::RaiseException', 'prim::Uninitialized', 'aten::lstm', 'aten::_cast_Int', 'aten::derive_index', 'aten::not', 'aten::tensor', 'aten::item', 'aten::_castFloat', 'prim::data', 'aten::format', 'aten::copy', 'aten::append', 'aten::FloatImplicit', 'prim::dtype', 'prim::shape', 'prim::TupleIndex', 'aten::dim', 'aten::warn', 'aten::is__', 'prim::unchecked_cast']

The model I'm using is from https://github.com/mlperf/inference/tree/master/v0.7/speech_recognition/rnnt/pytorch Pre-trained RNN-T model for MLPerf Inference https://zenodo.org/record/3662521

[Tomakko]

I have encountered a couple more missing op conversions:

NotImplementedError: The following operators are not implemented: ['aten::remainder', 'aten::copy_', 'prim::dtype', 'prim::unchecked_cast', 'aten::__is__', 'aten::silu_', 'aten::__isnot__', 'aten::append']

I am using a traced version of https://github.com/rwightman/efficientdet-pytorch

[neos-lc]

I have encountered the following missing op conversions:

The following operators are not implemented: ['aten::silu_', 'aten::copy_']

I'm trying to compile a YoloV5 model in AWS Sagemaker Neo. This is the full failure reason:

ClientError: InputConfiguration: TVM cannot convert the PyTorch model. Invalid model or input-shape mismatch. Make sure that inputs are lexically ordered and of the correct dimensionality. The following operators are not implemented: ['aten::silu_', 'aten::copy_']

[Kyrie-Zhao]

I am currently accelerating swin_transformer https://github.com/Kyrie-Zhao/Swin-Transformer/blob/main/models/swin_transformer.py which shows: NotImplementedError: The following operators are not implemented: ['aten::rand', 'aten::adaptive_avgpool1d', 'aten::roll', 'aten::floor']

Anyone can provide some hints about how to add & register these operators by yourself？Appreciate it.

[masahi]

Hi folks, support for aten::copy_ often comes up, but since this is a in-place mutation op, we don't have a good way to convert it correctly 100% of its use cases. So we decided not to support it for now. 'aten::silu' can be trivially supported by adding a one line `"aten::silu": self.silu` at https://github.com/apache/tvm/blob/9cf0245adb4cd61d07c278d7b7bcd01c9c5b9f8d/python/tvm/relay/frontend/pytorch.py#L2853

@Kyrie-Zhao, I didn't know about swin_transformer, looks like an interesting model. I'd love to get this model to work with TVM.

You're welcome to open a new issue. aten::rand and aten::roll might be tricky to convert to.

[chengven027]

Is the missing op aten::im2col and aten::col2im being implemented. I saw that #8443 gave up after half of the realization? @masahi Or is there any problem with it？

[masahi]

I don't know what happened to #8443. For im2col, we may be able to use sliding_window operator introduced in https://github.com/apache/tvm/pull/9816.

[nkhlS141]

{aten::pad, } op is missing