Use a circular buffer instead of growing a slice indefinitely and trimming it off from the left; no need to put pressure on the garbage collector;
For large number of samples, the sums may grow large, and we're computing a (relatively small) difference between two large numbers. Perhaps, keep the list of actual samples and the running sum, subtract the earliest sample and add the new one for each iteration. This way, the sum will always stay relatively small.
This may require updating the state type to a special struct that implements a circular buffer and the accumulated sum.