search

pytorch / benchmark

TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
BSD 3-Clause "New" or "Revised" License
817 stars 259 forks source link

[1/2] tritonbench bf16xint16 matmul template #2348

Open davidberard98 opened 2 days ago

davidberard98 commented 2 days ago

Stack from ghstack (oldest at bottom):

Overall context: Before looking further into the bf16xint4 matmul, I'm planning to look into a bf16xint16 matmul first. The idea of this matmul is that it will just be the same as a bf16xbf16 matmul, except the second operand needs to be casted from int16 to bf16 in the triton kernel before executing.

This PR: is NOT fully functional yet. It's just implemented this way to make review easier.

There's 3 kernels that will be benchmarked here:

  1. bf16xbf16 triton kernel - I've selected this kernel as the "baseline" because, ideally, we'd like the bf16xint16 kernel to be as close as possible to this kernel.
  2. bf16xint16 triton kernel - this is NOT implemented yet, will be implemented in the follow-up PR.
  3. bf16x(convert(int16 -> bf16)) triton kernel - i.e. convert the int16->bf16, write to global memory, and then run the bf16xbf16 kernel.

Differential Revision: D59234085

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

davidberard98 commented 1 day ago

@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.