ROCm / triton

Development repository for the Triton language and compiler
MIT License
80 stars 22 forks source link

[Upstream backend] [Pytorch UT]: `TypeError: function takes exactly 6 arguments (34 given)` #558

Closed jataylo closed 2 months ago

jataylo commented 2 months ago

Problem Description

This change https://github.com/openai/triton/commit/a0de891779975492a63f307f315b3063af2238f0 causes mass breakages with inductor+triton. It looks like the pack_metadata truncates the metadata tuple but the launch is expecting the full metadata.

Docker image: rocm/pytorch-private:tritoninductor-debug-args-issue

Pytorch branch: https://github.com/pytorch/pytorch/tree/rocm-inductor-hip-device

Triton branch: upstream

Traceback:

  File "rep.py", line 181, in call
    triton_red_fused__to_copy_abs_add_mul_remainder_xor_sum_0.run(buf0, arg1_1, buf1, s0, 0, 1, 432, triton_red_fused__to_copy_abs_add_mul_remainder_xor_sum_0_rnumel, grid=grid(432), stream=stream0)
  File "/tmp/pytorch/torch/_inductor/triton_heuristics.py", line 743, in run
    return launcher(
  File "<string>", line 13, in launcher
  File "/root/triton/python/triton/backends/amd/driver.py", line 417, in __call__
    self.launch(*args, **kwargs)
TypeError: function takes exactly 6 arguments (34 given)

Reproducer: TORCHINDUCTOR_COMPILE_THREADS=1 python ~/rep.py or TORCHINDUCTOR_COMPILE_THREADS=1 TORCHDYNAMO_REPRO_AFTER=aot PYTORCH_TEST_WITH_ROCM=1 python test_torchinductor.py -k "test_AllenaiLongformerBase_repro_cuda" --verbose

Operating System

-

CPU

-

GPU

AMD Instinct MI250X

ROCm Version

ROCm 6.0.0

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

zhanglx13 commented 2 months ago

https://github.com/pytorch/pytorch/commit/99059affb97a4d69c2c6b82e274f2fefb7ab765b