search

AnswerDotAI / gpu.cpp

A lightweight library for portable low-level GPU computation using WebGPU.
https://gpucpp.answer.ai
Apache License 2.0
3.75k stars 177 forks source link

Add autotune for matmul #52

Open junjihashimoto opened 3 months ago

junjihashimoto commented 3 months ago

This PR implements autotune for matmul. To skip unacceptable parameters with runtime-exception, it also implements additional exception-hander to catch exceptions on main-thread.

austinvhuang commented 2 months ago

Pretty excited to incorporate some autotuning functionality in the library.

On the exception handler, I need to have a closer look. What are the condition where the exception gets thrown? Some additional discussion on errors here https://github.com/AnswerDotAI/gpu.cpp/pull/49#issuecomment-2304744353 - can chat more in the discord chat.

I was thinking about a more generic mechanism for autotuning. There could be some higher order function that takes:

  1. A list of parameter sets to sweep through (a consumer of this could setup the product set of their parameters). Note the parameter set would be a value-level map (parameter name -> parameter -> value).
  2. a function that takes one specific parameter set, a number N of replications per parameter set.

The autotuner then sweeps through each parameter set and runs it N times. It then returns, for each parameter set, the N values of running times. Something like hyperfine https://github.com/sharkdp/hyperfine except operating on a parameter set and a function as input.