certik / fastGPT

Fast GPT-2 inference written in Fortran
MIT License
180 stars 16 forks source link

Implement fast_erf() and fast_gelu() #45

Open certik opened 1 year ago

certik commented 1 year ago

This approximates the erf() function directly.

To use it, apply the following patch:

--- a/gpt2.f90
+++ b/gpt2.f90
@@ -109,7 +109,7 @@ real(sp), intent(in) :: x(:,:), fc_w(:,:), fc_b(:), proj_w(:,:), proj_b(:)
 real(sp) :: y(size(x,1),size(x,2))
 !real(sp) :: a(4*size(x,1),size(x,2))
 !a = gelu(linear(x, fc_w, fc_b))
-y = linear(gelu(linear(x, fc_w, fc_b)), proj_w, proj_b)
+y = linear(fast_gelu(linear(x, fc_w, fc_b)), proj_w, proj_b)
 end function

 function attention(n_embd_head,n_seq,n_seq_x, q, k, v, mask) result(y)