Adamdad / kat

Kolmogorov-Arnold Transformer: A PyTorch Implementation with CUDA kernel
MIT License
492 stars 28 forks source link

RuntimeError: memory format option is only supported by strided tensors #7

Open eran88 opened 6 days ago

eran88 commented 6 days ago

Im only using the activation in a simple model: class KAT(nn.Module): def init( self,

) -> None:
    super().__init__()
    self.act1 = KAT_Group(mode="identity")

def forward(self, x: Tensor) -> Tensor:
    x = self.act1(x)

and getting the following error:

x = kat_rational_cu.rational_fwd_1dgroup(input, weight_numerator, weight_denominator, group) RuntimeError: memory format option is only supported by strided tensors

Adamdad commented 6 days ago

Could you please provide more context about the problem? For example, the environment version, system details, and any other relevant information. Also, are you using ROCm devices?

eran88 commented 5 days ago

PyTorch Version: 2.2.0+cu118 CUDA Available: True CUDA Version: 11.8 Number of CUDA devices: 8 CUDA Device Name: NVIDIA A100-SXM4-80GB ROCm Available: False Python Version: 3.10.9