Python backend "DLPack tensor is not contiguous error" on tensors with a 0 dim

[darintay]

Description pb_utils.Tensor.from_dlpack throws a "DLPack tensor is not contiguous" exception on some (but not all) tensors that have a 0 dimension.

If any dimension is 0, I don't see how the tensor could be non-contiguous.

(Originally I was also having issues with some dim=1 tensors, but it looks like that was fixed in https://github.com/triton-inference-server/python_backend/pull/281)

Triton Information What version of Triton are you using? 23.09

Are you using the Triton container or did you build it yourself? Built self

To Reproduce Here is my simple Python-backend model:

import torch
from torch.utils.dlpack import to_dlpack, from_dlpack
import triton_python_backend_utils as pb_utils
class TritonPythonModel:
    def execute(self, req):
        print("EXECUTE")

        SHAPES = [
                (0,),
                (1,),
                (2,),
                (0, 0),
                (0, 1),
                (1, 0),
                (1, 2),
                (2, 2),
                (0, 0, 0),
                (0, 0, 1),
                (0, 0, 2),
                (1, 0, 2),
                (1, 1, 2),
                (2, 1, 2),
                (2, 2, 1),
                (2, 2, 2),
            ]

        for shape in SHAPES:
            try:
                a = torch.ones(shape)
                a = a.contiguous()
                pb_utils.Tensor.from_dlpack("output", to_dlpack(a))
                print(f"PASS {shape=} {a.is_contiguous()=} {a.stride()=}")
            except Exception as e:
                print(f"FAIL {shape=} {a.is_contiguous()=} {a.stride()=} {e=}")

        print("EXECUTE DONE")

EXECUTE
PASS shape=(0,) a.is_contiguous()=True a.stride()=(1,)
PASS shape=(1,) a.is_contiguous()=True a.stride()=(1,)
PASS shape=(2,) a.is_contiguous()=True a.stride()=(1,)
PASS shape=(0, 0) a.is_contiguous()=True a.stride()=(1, 1)
PASS shape=(0, 1) a.is_contiguous()=True a.stride()=(1, 1)
PASS shape=(1, 0) a.is_contiguous()=True a.stride()=(1, 1)
PASS shape=(1, 2) a.is_contiguous()=True a.stride()=(2, 1)
PASS shape=(2, 2) a.is_contiguous()=True a.stride()=(2, 1)
FAIL shape=(0, 0, 0) a.is_contiguous()=True a.stride()=(1, 1, 1) e=TritonModelException('DLPack tensor is not contiguous. Only contiguous DLPack tensors that are stored in C-Order are supported.')
PASS shape=(0, 0, 1) a.is_contiguous()=True a.stride()=(1, 1, 1)
FAIL shape=(0, 0, 2) a.is_contiguous()=True a.stride()=(2, 2, 1) e=TritonModelException('DLPack tensor is not contiguous. Only contiguous DLPack tensors that are stored in C-Order are supported.')
FAIL shape=(1, 0, 2) a.is_contiguous()=True a.stride()=(2, 2, 1) e=TritonModelException('DLPack tensor is not contiguous. Only contiguous DLPack tensors that are stored in C-Order are supported.')
PASS shape=(1, 1, 2) a.is_contiguous()=True a.stride()=(2, 2, 1)
PASS shape=(2, 1, 2) a.is_contiguous()=True a.stride()=(2, 2, 1)
PASS shape=(2, 2, 1) a.is_contiguous()=True a.stride()=(2, 1, 1)
PASS shape=(2, 2, 2) a.is_contiguous()=True a.stride()=(4, 2, 1)
EXECUTE DONE

Using torch 1.13.1.

Expected behavior All tensors above should "PASS", as they are all contiguous.

[yinggeh]

Hi @darintay. I have forwarded the issue to the team. Also cc @Tabrizian

[yinggeh]

DLIS-6313