[refactor] Generalization of dual_gemm_silu_identity_mul

[warpuv]

Generalization of dual_gemm_silu_identity_mul to use custom activation function.

• renamed dual_gemm_silu_identity_mul to dual_gemm_lhs_activation_and_mul
• added a template parameter for the activation function to dual_gemm_lhs_activation_and_mul*.
• added customizable epilogue class (EpilogueLHSActivationAndMul) with the same template parameter.

What does this PR do?

Fixes #1140

Before submitting

• [x] Did you have fun?
  • Make sure you had fun coding 🙃
• [x] Did you read the contributor guideline?
• [x] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • [ ] N/A
• [ ] Did you make sure to update the docs?
  • [x] N/A
• [ ] Did you write any new necessary tests?
  • [x] N/A
• [ ] Did you update the changelog? (if needed)
  • [x] N/A

PR review

Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

[pansershrek]

@danthe3rd, @lw, @zyan0 Can you review this PR please? This update will open pass to implement fused GELUTanh for Gemma models

[danthe3rd]

Hi @pansershrek and @warpuv Thanks for opening this PR. In principle I'm happy to accept this line of contributions, however you should be aware that:

• this implementation is efficient only on Ampere GPUs (it will be super slow on Hopper)
• moving forward, the easiest way to do this would be to interleave B0 and B1 columns, so that the output of the GEMM is also interleaved, and the activation can still be fused in the epilogue in the same way (but the rest of the GEMM is untouched)

cc @tridao

[pansershrek]

Hi @danthe3rd ! Thank you for your replay, can you explain in more details your advice about B0 and B1 columns? We don't understand the difference in architecture that well as you :) .