certik / fastGPT

Fast GPT-2 inference written in Fortran
MIT License
180 stars 16 forks source link

This PR makes fastGPT work with current LFortran #55

Closed certik closed 5 months ago

certik commented 1 year ago

This is a Draft PR, since we will slowly remove all workarounds. All tests pass using GFortran. For LFortran, gpt2, test_basic_input works. test_more_inputs works sometimes (there is some bug there). test_chat only works inside ctest.

certik commented 1 year ago

Unfortunately, the bug https://github.com/lfortran/lfortran/issues/2069 is still there.

certik commented 1 year ago

The highest priority is probably this diff that makes the code to compile and segfault:

--- a/gpt2.f90
+++ b/gpt2.f90
@@ -52,23 +52,10 @@ end function
 function softmax(x) result(y)
 real(sp), intent(in) :: x(:,:)
 real(sp) :: y(size(x,1),size(x,2))
-integer :: i,j
-real(sp) :: s
+integer :: i
 do i = 1, size(x,2)
-    s = -1e10
-    do j = 1, size(x,1)
-        if (x(j,i) > s) s = x(j,i)
-    end do
-    do j = 1, size(x,1)
-        y(j,i) = exp(x(j,i) - s)
-    end do
-    s = 0
-    do j = 1, size(x,1)
-        s = s + y(j,i)
-    end do
-    do j = 1, size(x,1)
-        y(j,i) = y(j,i) / s
-    end do
+    y(:,i) = exp(x(:,i) - maxval(x(:,i)))
+    y(:,i) = y(:,i) / sum(y(:,i))
 end do
 end function
certik commented 5 months ago

Closing in favor of #72.