hidet-org / hidet

An open-source efficient deep learning framework/compiler, written in python.
https://hidet.org
Apache License 2.0
648 stars 52 forks source link

[Bug] binary arithmetic with CUDA scalar #95

Closed soodoshll closed 1 year ago

soodoshll commented 1 year ago

In https://github.com/hidet-org/hidet/blob/80a35d616f1c7a7444380a99d4fbe4fe07e11447/python/hidet/graph/ops/definitions/arithmetic.py#L486

It looks like in order to calculate a binary operator with a scalar on GPU, we need to first copy it to CPU (Is it expected? Will it affect the performance because of synchronization?). And the cpu() function raises an error saying we should first detach the variable in https://github.com/hidet-org/hidet/blob/80a35d616f1c7a7444380a99d4fbe4fe07e11447/python/hidet/graph/tensor.py#L486.

If we add detach here, here comes the second problem. It looks like the .numpy(or to_dlpack) method does not support a tensor with only one element and shape []. The error message is like:

  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/ops/definitions/arithmetic.py", line 529, in divide
    return binary_arithmetic(
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/ops/definitions/arithmetic.py", line 479, in binary_arithmetic
    x = x.dtype(x.item())
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/tensor.py", line 528, in item
    ret = self.squeeze(dims=list(range(len(self.shape)))).tolist()
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/tensor.py", line 488, in tolist
    ret = ret.numpy()
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/tensor.py", line 930, in numpy
    return np.from_dlpack(self)
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/tensor.py", line 467, in __dlpack__
    return to_dlpack(self)
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/impl/dlpack.py", line 251, in to_dlpack
    return DLManagedTensorContext(tensor).capsuled_dltensor()
  File "/home/su/accdiff/thirdparty/hidet/python/hidet/graph/impl/dlpack.py", line 229, in __init__
    data=tensor.storage.addr,
AttributeError: 'NoneType' object has no attribute 'addr'
yaoyaoding commented 1 year ago

Thanks @soodoshll!