Closed danielstumpp closed 2 years ago
Stage3 fully fused and passes tests w.r.t ground truth.
Verify that the dimensions are correct. Note that the scaling value had to be made very small to prevent gelu from always returning zero.
This includes some stage 4 dev that I used to start this branch. We'll still need additional verification of that as I mentioned on Slack.
Stage3 fully fused and passes tests w.r.t ground truth.
Verify that the dimensions are correct. Note that the scaling value had to be made very small to prevent gelu from always returning zero.
This includes some stage 4 dev that I used to start this branch. We'll still need additional verification of that as I mentioned on Slack.