Dear @vincefn,

I have removed the limit on the length of all R2R and R2C transforms and added DST R2R transforms in v1.3.3 on the develop branch. Is it possible to include this in the test suite of pyvkfft?

Best regards, Dmitrii

Great ! I'll find some time to expand the test suite and add the direct sine transform.

I did not remember there was a limitation for R2C ? I was already testing 1D R2C/C2R up to a size of 100000 (radix) or 10000 (non-radix)

There was a limitation that R2C only worked for even-length sequences. Now there is a new algorithm implementation that covers all cases (and can be forced on for even-length cases with forceCallbackVersionRealTransforms set to 1).

I started with the DST implementation.

With a quick 2D test on a 512x512 array, inplace transform, all forward DST transforms (type 1234) give the same result as scipy.

However when performing DST+iDST, only type 1 and 4 give the original array (also identical to scipy's result) back. Type 2 and 3 do not.

The type 2 and 3 inverse transforms should be fixed now. Thanks for finding out!

Now I have another issue with forward 2D out-of-place C2C transforms, for sizes >512 (radix) or some sizes > 262 (non radix). Testing further, dct and r2c also fail.

That's on my macbook pro M1 in OpenCL - I'm travelling so that's the only tests I can make for now:

...
pyopencl  C2C      (495,495) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=4.88e-07 ninf=6.48e-07 < 4.69e-06 (0.138) 1 iFFT: n2=4.28e-07 ninf=5.29e-07 < 4.69e-06 (0.113) 1   OK
pyopencl  C2C      (500,500) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=4.74e-07 ninf=5.32e-07 < 4.70e-06 (0.113) 1 iFFT: n2=4.01e-07 ninf=4.57e-07 < 4.70e-06 (0.097) 1   OK
pyopencl  C2C      (504,504) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=5.11e-07 ninf=6.45e-07 < 4.70e-06 (0.137) 1 iFFT: n2=4.25e-07 ninf=5.79e-07 < 4.70e-06 (0.123) 1   OK
pyopencl  C2C      (507,507) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=4.07e-07 ninf=4.49e-07 < 4.71e-06 (0.095) 1 iFFT: n2=3.95e-07 ninf=4.88e-07 < 4.71e-06 (0.104) 1   OK
pyopencl  C2C      (512,512) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=3.07e-07 ninf=4.36e-07 < 4.71e-06 (0.093) 1 iFFT: n2=3.07e-07 ninf=3.73e-07 < 4.71e-06 (0.079) 1   OK
pyopencl  C2C      (520,520) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.00e+00 < 4.72e-06 (212909.005) 1 iFFT: n2=4.29e-07 ninf=5.27e-07 < 4.72e-06 (0.112) 1 FAIL
pyopencl  C2C      (525,525) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.01e+00 < 4.72e-06 (213666.828) 1 iFFT: n2=4.84e-07 ninf=5.87e-07 < 4.72e-06 (0.124) 1 FAIL
pyopencl  C2C      (528,528) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=9.85e-01 < 4.72e-06 (208635.734) 1 iFFT: n2=4.17e-07 ninf=5.73e-07 < 4.72e-06 (0.121) 1 FAIL
pyopencl  C2C      (539,539) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.00e+00 < 4.73e-06 (211429.399) 1 iFFT: n2=4.06e-07 ninf=4.63e-07 < 4.73e-06 (0.098) 1 FAIL
pyopencl  C2C      (540,540) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=9.95e-01 < 4.73e-06 (210211.755) 1 iFFT: n2=5.51e-07 ninf=7.03e-07 < 4.73e-06 (0.148) 1 FAIL
pyopencl  C2C      (546,546) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.01e+00 < 4.74e-06 (212328.779) 1 iFFT: n2=4.21e-07 ninf=5.30e-07 < 4.74e-06 (0.112) 1 FAIL
...

and

pyopencl  C2C      (258,258) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=3.08e-07 ninf=3.69e-07 < 8.82e-06 (0.042) 1 iFFT: n2=3.05e-07 ninf=4.03e-07 < 8.82e-06 (0.046) 1   OK
pyopencl  C2C      (259,259) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=3.01e-07 ninf=3.23e-07 < 8.83e-06 (0.037) 1 iFFT: n2=3.36e-07 ninf=3.70e-07 < 8.83e-06 (0.042) 1   OK
pyopencl  C2C      (261,261) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.64e-07 ninf=2.49e-07 < 8.83e-06 (0.028) 1 iFFT: n2=2.68e-07 ninf=2.84e-07 < 8.83e-06 (0.032) 1   OK
pyopencl  C2C      (262,262) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=3.92e-07 ninf=3.99e-07 < 8.84e-06 (0.045) 1 iFFT: n2=3.97e-07 ninf=4.55e-07 < 8.84e-06 (0.051) 1   OK
pyopencl  C2C      (263,263) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.01e+00 < 8.84e-06 (113694.975) 1 iFFT: n2=9.87e-07 ninf=8.49e-07 < 8.84e-06 (0.096) 1 FAIL
pyopencl  C2C      (265,265) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.66e-07 ninf=2.83e-07 < 8.85e-06 (0.032) 1 iFFT: n2=2.66e-07 ninf=3.17e-07 < 8.85e-06 (0.036) 1   OK
pyopencl  C2C      (266,266) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.76e-07 ninf=3.33e-07 < 8.85e-06 (0.038) 1 iFFT: n2=2.70e-07 ninf=3.12e-07 < 8.85e-06 (0.035) 1   OK
pyopencl  C2C      (267,267) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.90e-07 ninf=3.74e-07 < 8.85e-06 (0.042) 1 iFFT: n2=2.82e-07 ninf=3.62e-07 < 8.85e-06 (0.041) 1   OK
pyopencl  C2C      (268,268) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.79e-07 ninf=3.20e-07 < 8.86e-06 (0.036) 1 iFFT: n2=2.82e-07 ninf=3.23e-07 < 8.86e-06 (0.036) 1   OK
pyopencl  C2C      (269,269) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.00e+00 < 8.86e-06 (113412.991) 1 iFFT: n2=9.61e-07 ninf=9.27e-07 < 8.86e-06 (0.105) 1 FAIL
pyopencl  C2C      (271,271) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=4.57e-07 ninf=5.28e-07 < 8.87e-06 (0.060) 1 iFFT: n2=4.56e-07 ninf=5.35e-07 < 8.87e-06 (0.060) 1   OK
pyopencl  C2C      (272,272) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.52e-07 ninf=2.84e-07 < 8.87e-06 (0.032) 1 iFFT: n2=2.64e-07 ninf=2.87e-07 < 8.87e-06 (0.032) 1   OK
pyopencl  C2C      (274,274) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.01e+00 < 8.88e-06 (113942.442) 1 iFFT: n2=9.32e-07 ninf=1.02e-06 < 8.88e-06 (0.115) 1 FAIL
pyopencl  C2C      (276,276) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.38e-07 ninf=2.72e-07 < 8.88e-06 (0.031) 1 iFFT: n2=2.43e-07 ninf=2.65e-07 < 8.88e-06 (0.030) 1   OK
pyopencl  C2C      (277,277) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=9.94e-01 < 8.88e-06 (111881.789) 1 iFFT: n2=9.90e-07 ninf=1.11e-06 < 8.88e-06 (0.125) 1 FAIL
pyopencl  C2C      (278,278) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=1.00e+00 ninf=1.01e+00 < 8.89e-06 (113079.024) 1 iFFT: n2=9.71e-07 ninf=1.04e-06 < 8.89e-06 (0.116) 1 FAIL
pyopencl  C2C      (279,279) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=2.96e-07 ninf=3.50e-07 < 8.89e-06 (0.039) 1 iFFT: n2=2.88e-07 ninf=2.92e-07 < 8.89e-06 (0.033) 1   OK
pyopencl  C2C      (281,281) axes=      None ndim=   2  complex64 lut=False inplace=0  norm=   1 C   FFT: n2=4.30e-07 ninf=5.15e-07 < 8.90e-06 (0.058) 1 iFFT: n2=4.24e-07 ninf=4.99e-07 < 8.90e-06 (0.056) 1   OK

Well, that was a stupid mistake that when I moved tempBuffer allocation for intermediate system stores in multiupload algorithm, I had an incorrect parameter for setting buffer layout. It should be fixed (all systems you mentioned pass on M1).

Ok, looks better !

Now in the quick tests, a number of DST1 inverse transforms fail, when the number of transform dimensions is odd (?):

pyopencl DST1          (30,) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.52e-07 ninf=1.77e-07 < 2.74e-06 (0.065) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.572) 0 FAIL
pyopencl DST1          (30,) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=9.87e-08 ninf=6.89e-08 < 2.74e-06 (0.025) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.657) 1 FAIL
pyopencl DST1          (30,) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.39e-07 ninf=1.13e-07 < 2.74e-06 (0.041) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.668) 0 FAIL
pyopencl DST1          (30,) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.31e-07 ninf=1.69e-07 < 2.74e-06 (0.062) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.662) 1 FAIL
pyopencl DST1         (2,30) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.64e-07 ninf=1.85e-07 < 2.74e-06 (0.068) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.675) 0 FAIL
pyopencl DST1         (2,30) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.77e-07 ninf=1.60e-07 < 2.74e-06 (0.059) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.627) 1 FAIL
pyopencl DST1         (2,30) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.62e-07 ninf=1.94e-07 < 2.74e-06 (0.071) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.586) 0 FAIL
pyopencl DST1         (2,30) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.88e-07 ninf=2.46e-07 < 2.74e-06 (0.090) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.623) 1 FAIL
pyopencl DST1        (30,30) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.26e-07 ninf=1.97e-07 < 3.48e-06 (0.057) 0 iFFT: n2=2.39e-07 ninf=2.25e-07 < 3.48e-06 (0.065) 0   OK
pyopencl DST1        (30,30) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.26e-07 ninf=2.11e-07 < 3.48e-06 (0.061) 1 iFFT: n2=2.54e-07 ninf=2.49e-07 < 3.48e-06 (0.072) 1   OK
pyopencl DST1        (30,30) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.32e-07 ninf=2.72e-07 < 3.48e-06 (0.078) 0 iFFT: n2=2.57e-07 ninf=3.71e-07 < 3.48e-06 (0.107) 0   OK
pyopencl DST1        (30,30) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.24e-07 ninf=2.49e-07 < 3.48e-06 (0.072) 1 iFFT: n2=2.47e-07 ninf=2.32e-07 < 3.48e-06 (0.067) 1   OK
pyopencl DST2        (30,30) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.63e-07 ninf=3.46e-07 < 3.48e-06 (0.099) 0 iFFT: n2=3.04e-07 ninf=3.16e-07 < 3.48e-06 (0.091) 0   OK
pyopencl DST1       (2,2,30) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.77e-07 ninf=2.30e-07 < 2.74e-06 (0.084) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.683) 0 FAIL
pyopencl DST1       (2,2,30) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.67e-07 ninf=1.64e-07 < 2.74e-06 (0.060) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.675) 1 FAIL
pyopencl DST1       (2,2,30) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.69e-07 ninf=1.62e-07 < 2.74e-06 (0.059) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.659) 0 FAIL
pyopencl DST1       (2,2,30) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.75e-07 ninf=1.90e-07 < 2.74e-06 (0.069) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.640) 1 FAIL
pyopencl DST1      (2,30,30) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.17e-07 ninf=2.49e-07 < 3.48e-06 (0.072) 0 iFFT: n2=2.46e-07 ninf=2.87e-07 < 3.48e-06 (0.083) 0   OK
pyopencl DST1      (2,30,30) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.26e-07 ninf=2.83e-07 < 3.48e-06 (0.082) 1 iFFT: n2=2.39e-07 ninf=2.50e-07 < 3.48e-06 (0.072) 1   OK
pyopencl DST1      (2,30,30) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.21e-07 ninf=2.28e-07 < 3.48e-06 (0.065) 0 iFFT: n2=2.53e-07 ninf=2.72e-07 < 3.48e-06 (0.078) 0   OK
pyopencl DST1      (2,30,30) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.12e-07 ninf=2.32e-07 < 3.48e-06 (0.067) 1 iFFT: n2=2.43e-07 ninf=2.85e-07 < 3.48e-06 (0.082) 1   OK
pyopencl DST1     (30,30,30) axes=      None ndim=   3    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.71e-07 ninf=3.66e-07 < 4.22e-06 (0.087) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.078) 0 FAIL
pyopencl DST1     (30,30,30) axes=      None ndim=   3    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.66e-07 ninf=3.76e-07 < 4.22e-06 (0.089) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.059) 1 FAIL
pyopencl DST1     (30,30,30) axes=      None ndim=   3    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.66e-07 ninf=3.01e-07 < 4.22e-06 (0.071) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.087) 0 FAIL
pyopencl DST1     (30,30,30) axes=      None ndim=   3    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.69e-07 ninf=3.58e-07 < 4.22e-06 (0.085) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474418.981) 1 FAIL
pyopencl DST1     (2,2,2,30) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.64e-07 ninf=1.79e-07 < 2.74e-06 (0.065) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.671) 0 FAIL
pyopencl DST1     (2,2,2,30) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.71e-07 ninf=1.60e-07 < 2.74e-06 (0.058) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.696) 1 FAIL
pyopencl DST1     (2,2,2,30) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.62e-07 ninf=1.93e-07 < 2.74e-06 (0.071) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.634) 0 FAIL
pyopencl DST1     (2,2,2,30) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.66e-07 ninf=1.91e-07 < 2.74e-06 (0.070) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 2.74e-06 (730310.600) 1 FAIL
pyopencl DST1    (2,2,30,30) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.23e-07 ninf=2.71e-07 < 3.48e-06 (0.078) 0 iFFT: n2=2.51e-07 ninf=2.93e-07 < 3.48e-06 (0.084) 0   OK
pyopencl DST1    (2,2,30,30) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.20e-07 ninf=1.91e-07 < 3.48e-06 (0.055) 1 iFFT: n2=2.44e-07 ninf=2.64e-07 < 3.48e-06 (0.076) 1   OK
pyopencl DST1    (2,2,30,30) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.24e-07 ninf=2.70e-07 < 3.48e-06 (0.078) 0 iFFT: n2=2.54e-07 ninf=3.07e-07 < 3.48e-06 (0.088) 0   OK
pyopencl DST1    (2,2,30,30) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.17e-07 ninf=2.09e-07 < 3.48e-06 (0.060) 1 iFFT: n2=2.41e-07 ninf=2.30e-07 < 3.48e-06 (0.066) 1   OK
pyopencl DST1   (2,30,30,30) axes=      None ndim=   3    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.67e-07 ninf=2.77e-07 < 4.22e-06 (0.066) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.068) 0 FAIL
pyopencl DST1   (2,30,30,30) axes=      None ndim=   3    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.67e-07 ninf=3.28e-07 < 4.22e-06 (0.078) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.041) 1 FAIL
pyopencl DST1   (2,30,30,30) axes=      None ndim=   3    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.68e-07 ninf=2.86e-07 < 4.22e-06 (0.068) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.040) 0 FAIL
pyopencl DST1   (2,30,30,30) axes=      None ndim=   3    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.67e-07 ninf=3.16e-07 < 4.22e-06 (0.075) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.22e-06 (474419.068) 1 FAIL
pyopencl DST1  (30,30,30,30) axes=      None ndim=   4    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.10e-07 ninf=3.22e-07 < 4.95e-06 (0.065) 0 iFFT: n2=3.77e-07 ninf=4.13e-07 < 4.95e-06 (0.083) 0   OK
pyopencl DST1  (30,30,30,30) axes=      None ndim=   4    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.10e-07 ninf=3.30e-07 < 4.95e-06 (0.067) 1 iFFT: n2=3.77e-07 ninf=4.74e-07 < 4.95e-06 (0.096) 1   OK
pyopencl DST1  (30,30,30,30) axes=      None ndim=   4    float32 lut=True inplace=1  norm=   1 C   FFT: n2=3.11e-07 ninf=3.55e-07 < 4.95e-06 (0.072) 0 iFFT: n2=3.77e-07 ninf=4.47e-07 < 4.95e-06 (0.090) 0   OK
pyopencl DST1  (30,30,30,30) axes=      None ndim=   4    float32 lut=True inplace=0  norm=   1 C   FFT: n2=3.11e-07 ninf=3.15e-07 < 4.95e-06 (0.064) 1 iFFT: n2=3.77e-07 ninf=3.97e-07 < 4.95e-06 (0.080) 1   OK
pyopencl DST1          (34,) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.25e-07 ninf=8.18e-08 < 5.53e-06 (0.015) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.957) 0 FAIL
pyopencl DST1          (34,) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.63e-07 ninf=1.76e-07 < 5.53e-06 (0.032) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.941) 1 FAIL
pyopencl DST1          (34,) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=6.69e-08 ninf=7.77e-08 < 5.53e-06 (0.014) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.960) 0 FAIL
pyopencl DST1          (34,) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=6.20e-08 ninf=8.10e-08 < 5.53e-06 (0.015) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.992) 1 FAIL
pyopencl DST1         (2,34) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.90e-07 ninf=1.93e-07 < 5.53e-06 (0.035) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.917) 0 FAIL
pyopencl DST1         (2,34) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.66e-07 ninf=1.72e-07 < 5.53e-06 (0.031) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.946) 1 FAIL
pyopencl DST1         (2,34) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.08e-07 ninf=1.34e-07 < 5.53e-06 (0.024) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.936) 0 FAIL
pyopencl DST1         (2,34) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=9.14e-08 ninf=1.04e-07 < 5.53e-06 (0.019) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.939) 1 FAIL
pyopencl DST1        (34,34) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.48e-07 ninf=3.10e-07 < 7.06e-06 (0.044) 0 iFFT: n2=2.30e-07 ninf=2.47e-07 < 7.06e-06 (0.035) 0   OK
pyopencl DST1        (34,34) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.46e-07 ninf=2.70e-07 < 7.06e-06 (0.038) 1 iFFT: n2=2.31e-07 ninf=3.51e-07 < 7.06e-06 (0.050) 1   OK
pyopencl DST1        (34,34) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.23e-07 ninf=1.26e-07 < 7.06e-06 (0.018) 0 iFFT: n2=1.31e-07 ninf=1.27e-07 < 7.06e-06 (0.018) 0   OK
pyopencl DST1        (34,34) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.17e-07 ninf=1.24e-07 < 7.06e-06 (0.018) 1 iFFT: n2=1.23e-07 ninf=1.40e-07 < 7.06e-06 (0.020) 1   OK
pyopencl DST1       (2,2,34) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.00e-07 ninf=2.55e-07 < 5.53e-06 (0.046) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.965) 0 FAIL
pyopencl DST1       (2,2,34) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.76e-07 ninf=2.10e-07 < 5.53e-06 (0.038) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.932) 1 FAIL
pyopencl DST1       (2,2,34) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=9.21e-08 ninf=9.45e-08 < 5.53e-06 (0.017) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.964) 0 FAIL
pyopencl DST1       (2,2,34) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=8.90e-08 ninf=1.15e-07 < 5.53e-06 (0.021) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.954) 1 FAIL
pyopencl DST1      (2,34,34) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.35e-07 ninf=2.54e-07 < 7.06e-06 (0.036) 0 iFFT: n2=2.20e-07 ninf=1.97e-07 < 7.06e-06 (0.028) 0   OK
pyopencl DST1      (2,34,34) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.53e-07 ninf=2.60e-07 < 7.06e-06 (0.037) 1 iFFT: n2=2.30e-07 ninf=2.66e-07 < 7.06e-06 (0.038) 1   OK
pyopencl DST1      (2,34,34) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.17e-07 ninf=1.48e-07 < 7.06e-06 (0.021) 0 iFFT: n2=1.24e-07 ninf=1.22e-07 < 7.06e-06 (0.017) 0   OK
pyopencl DST1      (2,34,34) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.19e-07 ninf=1.10e-07 < 7.06e-06 (0.016) 1 iFFT: n2=1.25e-07 ninf=1.42e-07 < 7.06e-06 (0.020) 1   OK
pyopencl DST1     (34,34,34) axes=      None ndim=   3    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.06e-07 ninf=3.29e-07 < 8.59e-06 (0.038) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 8.59e-06 (232708.671) 0 FAIL
pyopencl DST1     (34,34,34) axes=      None ndim=   3    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.06e-07 ninf=3.16e-07 < 8.59e-06 (0.037) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 8.59e-06 (232708.656) 1 FAIL
pyopencl DST1     (34,34,34) axes=      None ndim=   3    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.43e-07 ninf=1.79e-07 < 8.59e-06 (0.021) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 8.59e-06 (232708.678) 0 FAIL
pyopencl DST1     (34,34,34) axes=      None ndim=   3    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.42e-07 ninf=1.79e-07 < 8.59e-06 (0.021) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 8.59e-06 (232708.678) 1 FAIL
pyopencl DST1     (2,2,2,34) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.69e-07 ninf=1.93e-07 < 5.53e-06 (0.035) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.929) 0 FAIL
pyopencl DST1     (2,2,2,34) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.76e-07 ninf=2.11e-07 < 5.53e-06 (0.038) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.963) 1 FAIL
pyopencl DST1     (2,2,2,34) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=9.98e-08 ninf=1.00e-07 < 5.53e-06 (0.018) 0 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.955) 0 FAIL
pyopencl DST1     (2,2,2,34) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=9.27e-08 ninf=8.49e-08 < 5.53e-06 (0.015) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.53e-06 (361566.959) 1 FAIL
pyopencl DST1    (2,2,34,34) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.48e-07 ninf=2.90e-07 < 7.06e-06 (0.041) 0 iFFT: n2=2.28e-07 ninf=2.60e-07 < 7.06e-06 (0.037) 0   OK
pyopencl DST1    (2,2,34,34) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.43e-07 ninf=2.67e-07 < 7.06e-06 (0.038) 1 iFFT: n2=2.25e-07 ninf=2.43e-07 < 7.06e-06 (0.034) 1   OK
pyopencl DST1    (2,2,34,34) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.17e-07 ninf=1.52e-07 < 7.06e-06 (0.021) 0 iFFT: n2=1.25e-07 ninf=1.35e-07 < 7.06e-06 (0.019) 0   OK
pyopencl DST1    (2,2,34,34) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.21e-07 ninf=1.21e-07 < 7.06e-06 (0.017) 1 iFFT: n2=1.23e-07 ninf=1.32e-07 < 7.06e-06 (0.019) 1   OK

The other DST's look fine.

OK, I extended the R2C/DCT/DST quick tests to longer sizes, and in addition to the above, I have new failures:

For C2R on 2D transform with size 2988:

pyopencl  R2C       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=7.18e-07 ninf=9.20e-07 < 7.48e-06 (0.123) 1 iFFT: n2=7.16e-07 ninf=1.54e-06 < 7.48e-06 (0.206) 0   OK
pyopencl  R2C       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=7.33e-07 ninf=7.64e-07 < 7.48e-06 (0.102) 1 iFFT: n2=7.73e-07 ninf=1.64e-06 < 7.48e-06 (0.219) 0   OK
pyopencl  R2C     (2,2988+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=2.29e-07 ninf=2.67e-07 < 7.48e-06 (0.036) 0 iFFT: n2=2.23e-07 ninf=4.80e-07 < 7.48e-06 (0.064) 0   OK
pyopencl  R2C     (2,2988+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.27e-07 ninf=2.37e-07 < 7.48e-06 (0.032) 0 iFFT: n2=2.47e-07 ninf=5.06e-07 < 7.48e-06 (0.068) 0   OK
pyopencl  R2C       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=2.28e-07 ninf=2.18e-07 < 7.48e-06 (0.029) 1 iFFT: n2=2.26e-07 ninf=4.62e-07 < 7.48e-06 (0.062) 0   OK
pyopencl  R2C       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.27e-07 ninf=2.36e-07 < 7.48e-06 (0.032) 1 iFFT: n2=2.45e-07 ninf=5.93e-07 < 7.48e-06 (0.079) 0   OK
pyopencl  R2C  (2988,2988+2) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   0 C   FFT: n2=1.30e-06 ninf=1.57e-06 < 1.10e-05 (0.143) 0 iFFT: n2=1.22e+00 ninf=3.65e+00 < 1.10e-05 (333382.766) 0 FAIL
pyopencl  R2C  (2988+2,2988) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   0 F   FFT: n2=1.30e-06 ninf=1.56e-06 < 1.10e-05 (0.142) 0 iFFT: n2=1.23e+00 ninf=3.78e+00 < 1.10e-05 (345126.973) 0 FAIL
pyopencl  R2C  (2988,2988+2) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.30e-06 ninf=1.44e-06 < 1.10e-05 (0.132) 0 iFFT: n2=3.87e+01 ninf=1.58e+02 < 1.10e-05 (14468929.632) 0 FAIL
pyopencl  R2C  (2988+2,2988) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 F   FFT: n2=1.30e-06 ninf=1.38e-06 < 1.10e-05 (0.126) 0 iFFT: n2=3.87e+01 ninf=1.53e+02 < 1.10e-05 (13947169.529) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   0 C   FFT: n2=1.30e-06 ninf=1.37e-06 < 1.10e-05 (0.125) 1 iFFT: n2=1.23e+00 ninf=3.60e+00 < 1.10e-05 (328922.944) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   0 F   FFT: n2=1.30e-06 ninf=1.39e-06 < 1.10e-05 (0.127) 1 iFFT: n2=1.23e+00 ninf=3.49e+00 < 1.10e-05 (318629.321) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.30e-06 ninf=1.40e-06 < 1.10e-05 (0.128) 1 iFFT: n2=3.87e+01 ninf=1.65e+02 < 1.10e-05 (15085335.601) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 F   FFT: n2=1.30e-06 ninf=1.41e-06 < 1.10e-05 (0.129) 1 iFFT: n2=3.86e+01 ninf=1.62e+02 < 1.10e-05 (14793412.441) 0 FAIL
pyopencl  R2C  (2988,2988+2) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   0 C   FFT: n2=4.22e-07 ninf=4.50e-07 < 1.10e-05 (0.041) 0 iFFT: n2=1.22e+00 ninf=3.42e+00 < 1.10e-05 (312648.244) 0 FAIL
pyopencl  R2C  (2988+2,2988) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   0 F   FFT: n2=4.22e-07 ninf=4.64e-07 < 1.10e-05 (0.042) 0 iFFT: n2=1.23e+00 ninf=3.30e+00 < 1.10e-05 (301510.065) 0 FAIL
pyopencl  R2C  (2988,2988+2) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=4.22e-07 ninf=5.29e-07 < 1.10e-05 (0.048) 0 iFFT: n2=3.87e+01 ninf=1.69e+02 < 1.10e-05 (15413907.190) 0 FAIL
pyopencl  R2C  (2988+2,2988) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 F   FFT: n2=4.22e-07 ninf=4.92e-07 < 1.10e-05 (0.045) 0 iFFT: n2=3.87e+01 ninf=1.60e+02 < 1.10e-05 (14586527.948) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   0 C   FFT: n2=4.22e-07 ninf=4.39e-07 < 1.10e-05 (0.040) 1 iFFT: n2=1.23e+00 ninf=3.52e+00 < 1.10e-05 (321127.622) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   0 F   FFT: n2=4.22e-07 ninf=5.05e-07 < 1.10e-05 (0.046) 1 iFFT: n2=1.23e+00 ninf=3.46e+00 < 1.10e-05 (316078.377) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=4.22e-07 ninf=4.18e-07 < 1.10e-05 (0.038) 1 iFFT: n2=3.87e+01 ninf=1.64e+02 < 1.10e-05 (14935897.735) 0 FAIL
pyopencl  R2C    (2988,2988) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 F   FFT: n2=4.22e-07 ninf=4.46e-07 < 1.10e-05 (0.041) 1 iFFT: n2=3.87e+01 ninf=1.60e+02 < 1.10e-05 (14616991.226) 0 FAIL
pyopencl  R2C       (4200+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.43e-07 ninf=4.08e-07 < 7.62e-06 (0.054) 0 iFFT: n2=3.37e-07 ninf=7.76e-07 < 7.62e-06 (0.102) 0   OK
pyopencl  R2C       (4200+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.48e-07 ninf=4.33e-07 < 7.62e-06 (0.057) 0 iFFT: n2=3.83e-07 ninf=8.88e-07 < 7.62e-06 (0.117) 0   OK
pyopencl  R2C        (4200,) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.47e-07 ninf=4.31e-07 < 3.81e-06 (0.113) 1 iFFT: n2=3.33e-07 ninf=7.34e-07 < 3.81e-06 (0.193) 0   OK
pyopencl  R2C        (4200,) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.51e-07 ninf=4.50e-07 < 3.81e-06 (0.118) 1 iFFT: n2=3.83e-07 ninf=1.09e-06 < 3.81e-06 (0.287) 0   OK
pyopencl  R2C       (4200+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.45e-07 ninf=1.66e-07 < 7.62e-06 (0.022) 0 iFFT: n2=1.44e-07 ninf=3.65e-07 < 7.62e-06 (0.048) 0   OK

DCT2 (same on DST2) on some forward transforms:

pyopencl DCT1      (808,808) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=4.06e-07 ninf=3.85e-07 < 9.81e-06 (0.039) 0 iFFT: n2=3.70e-07 ninf=3.95e-07 < 9.81e-06 (0.040) 0   OK
pyopencl DCT1      (808,808) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=4.06e-07 ninf=4.17e-07 < 9.81e-06 (0.043) 1 iFFT: n2=3.71e-07 ninf=3.95e-07 < 9.81e-06 (0.040) 1   OK
pyopencl DCT2      (808,808) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=9.96e+00 ninf=1.19e+01 < 9.81e-06 (1210716.679) 0 iFFT: n2=2.71e-07 ninf=3.28e-07 < 9.81e-06 (0.033) 0 FAIL
pyopencl DCT2      (808,808) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=9.97e+00 ninf=1.41e+01 < 9.81e-06 (1434801.514) 1 iFFT: n2=2.71e-07 ninf=2.88e-07 < 9.81e-06 (0.029) 1 FAIL
pyopencl DCT2      (808,808) axes=      None ndim=   2    float32 lut=True inplace=1  norm=   1 C   FFT: n2=9.98e+00 ninf=1.11e+01 < 9.81e-06 (1126029.765) 0 iFFT: n2=2.48e-07 ninf=2.50e-07 < 9.81e-06 (0.025) 0 FAIL
pyopencl DCT2      (808,808) axes=      None ndim=   2    float32 lut=True inplace=0  norm=   1 C   FFT: n2=9.97e+00 ninf=1.16e+01 < 9.81e-06 (1180203.970) 1 iFFT: n2=2.48e-07 ninf=2.47e-07 < 9.81e-06 (0.025) 1 FAIL
pyopencl DCT3      (808,808) axes=      None ndim=   2    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.63e-07 ninf=3.04e-07 < 9.81e-06 (0.031) 0 iFFT: n2=3.18e-07 ninf=3.37e-07 < 9.81e-06 (0.034) 0   OK
pyopencl DCT3      (808,808) axes=      None ndim=   2    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.62e-07 ninf=2.64e-07 < 9.81e-06 (0.027) 1 iFFT: n2=3.18e-07 ninf=2.77e-07 < 9.81e-06 (0.028) 1   OK
...
pyopencl DCT2       (13002,) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.44e+01 < 8.11e-06 (1776093.108) 0 iFFT: n2=2.28e-07 ninf=2.33e-07 < 8.11e-06 (0.029) 0 FAIL
pyopencl DCT2       (13002,) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.20e+01 < 8.11e-06 (1476360.531) 1 iFFT: n2=2.31e-07 ninf=2.07e-07 < 8.11e-06 (0.026) 1 FAIL
pyopencl DCT2       (13002,) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.69e+01 < 8.11e-06 (2078244.454) 0 iFFT: n2=1.68e-07 ninf=1.83e-07 < 8.11e-06 (0.023) 0 FAIL
pyopencl DCT2       (13002,) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.40e+01 < 8.11e-06 (1723369.395) 1 iFFT: n2=1.67e-07 ninf=1.85e-07 < 8.11e-06 (0.023) 1 FAIL
...
pyopencl DCT2      (2,13002) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.17e+01 < 8.11e-06 (1437117.729) 0 iFFT: n2=2.31e-07 ninf=2.97e-07 < 8.11e-06 (0.037) 0 FAIL
pyopencl DCT2      (2,13002) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.50e+01 < 8.11e-06 (1848426.997) 1 iFFT: n2=2.31e-07 ninf=2.58e-07 < 8.11e-06 (0.032) 1 FAIL
pyopencl DCT2      (2,13002) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.31e+01 < 8.11e-06 (1619360.586) 0 iFFT: n2=1.66e-07 ninf=1.76e-07 < 8.11e-06 (0.022) 0 FAIL
pyopencl DCT2      (2,13002) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.54e+01 < 8.11e-06 (1903673.328) 1 iFFT: n2=1.65e-07 ninf=1.91e-07 < 8.11e-06 (0.024) 1 FAIL
...
pyopencl DCT2    (2,2,13002) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.36e+01 < 8.11e-06 (1670713.037) 0 iFFT: n2=2.31e-07 ninf=2.68e-07 < 8.11e-06 (0.033) 0 FAIL
pyopencl DCT2    (2,2,13002) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.59e+01 < 8.11e-06 (1954555.923) 1 iFFT: n2=2.29e-07 ninf=2.39e-07 < 8.11e-06 (0.029) 1 FAIL
pyopencl DCT2    (2,2,13002) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.39e+01 ninf=1.33e+01 < 8.11e-06 (1635548.039) 0 iFFT: n2=1.67e-07 ninf=2.13e-07 < 8.11e-06 (0.026) 0 FAIL
pyopencl DCT2    (2,2,13002) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.39e+01 ninf=1.62e+01 < 8.11e-06 (1992159.136) 1 iFFT: n2=1.67e-07 ninf=1.69e-07 < 8.11e-06 (0.021) 1 FAIL

I pushed the changes required to support DCT, DST, and expanding the tests for longer sizes.

I also added the support for odd lengths for the R2C/C2R transforms, which are also supported with the tests. I just set forceCallbackVersionRealTransforms to 1 for odd-sized fast axis R2C transforms, and this gives the same result as scipy.

Regarding the above errors for the 2D C2R transforms, using the systematic test I can see the error appear for the first time for a size of 263 (and only on non-radix transforms, at least up to 1000) - this is still on my macbook M1:

pyvkfft-test --systematic --nproc 4 --r2c --ndim 2 --range 1 1000 --inplace
...
pyopencl  R2C    (256,256+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.35e-07 ninf=2.98e-07 < 8.82e-06 (0.034) 0 iFFT: n2=2.40e-07 ninf=6.56e-07 < 8.82e-06 (0.074) 0   OK
pyopencl  R2C    (257,257+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.59e-07 ninf=4.59e-07 < 8.82e-06 (0.052) 0 iFFT: n2=4.34e-07 ninf=1.07e-06 < 8.82e-06 (0.122) 0   OK
pyopencl  R2C    (258,258+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=3.02e-07 ninf=2.97e-07 < 4.41e-06 (0.067) 0 iFFT: n2=3.00e-07 ninf=8.34e-07 < 4.41e-06 (0.189) 0   OK
pyopencl  R2C    (259,259+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.97e-07 ninf=4.40e-07 < 4.41e-06 (0.100) 0 iFFT: n2=3.21e-07 ninf=8.34e-07 < 4.41e-06 (0.189) 0   OK
pyopencl  R2C    (260,260+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=3.63e-07 ninf=3.92e-07 < 8.83e-06 (0.044) 0 iFFT: n2=3.44e-07 ninf=8.94e-07 < 8.83e-06 (0.101) 0   OK
pyopencl  R2C    (261,261+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.60e-07 ninf=2.51e-07 < 8.83e-06 (0.028) 0 iFFT: n2=2.47e-07 ninf=6.56e-07 < 8.83e-06 (0.074) 0   OK
pyopencl  R2C    (262,262+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=3.51e-07 ninf=3.97e-07 < 4.42e-06 (0.090) 0 iFFT: n2=3.55e-07 ninf=8.79e-07 < 4.42e-06 (0.199) 0   OK
pyopencl  R2C    (263,263+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=1.02e-06 ninf=1.11e-06 < 4.42e-06 (0.250) 0 iFFT: n2=1.15e+01 ninf=4.04e+01 < 4.42e-06 (9150231.603) 0 FAIL
pyopencl  R2C    (264,264+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.38e-07 ninf=5.58e-07 < 8.84e-06 (0.063) 0 iFFT: n2=4.05e-07 ninf=1.32e-06 < 8.84e-06 (0.149) 0   OK
pyopencl  R2C    (265,265+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.59e-07 ninf=2.82e-07 < 8.85e-06 (0.032) 0 iFFT: n2=2.44e-07 ninf=6.56e-07 < 8.85e-06 (0.074) 0   OK
pyopencl  R2C    (266,266+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.78e-07 ninf=3.60e-07 < 8.85e-06 (0.041) 0 iFFT: n2=2.68e-07 ninf=7.15e-07 < 8.85e-06 (0.081) 0   OK
pyopencl  R2C    (267,267+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.83e-07 ninf=2.96e-07 < 8.85e-06 (0.033) 0 iFFT: n2=2.59e-07 ninf=6.56e-07 < 8.85e-06 (0.074) 0   OK
pyopencl  R2C    (268,268+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.78e-07 ninf=3.41e-07 < 4.43e-06 (0.077) 0 iFFT: n2=2.79e-07 ninf=7.75e-07 < 4.43e-06 (0.175) 0   OK
pyopencl  R2C    (269,269+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=9.62e-07 ninf=9.91e-07 < 4.43e-06 (0.224) 0 iFFT: n2=1.16e+01 ninf=4.71e+01 < 4.43e-06 (10642407.850) 0 FAIL
pyopencl  R2C    (270,270+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=5.02e-07 ninf=5.30e-07 < 8.86e-06 (0.060) 0 iFFT: n2=5.04e-07 ninf=1.19e-06 < 8.86e-06 (0.135) 0   OK
pyopencl  R2C    (271,271+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.55e-07 ninf=5.12e-07 < 8.87e-06 (0.058) 0 iFFT: n2=4.16e-07 ninf=1.01e-06 < 8.87e-06 (0.114) 0   OK
pyopencl  R2C    (272,272+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.46e-07 ninf=3.20e-07 < 8.87e-06 (0.036) 0 iFFT: n2=2.66e-07 ninf=7.15e-07 < 8.87e-06 (0.081) 0   OK
pyopencl  R2C    (273,273+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=3.79e-07 ninf=4.70e-07 < 8.87e-06 (0.053) 0 iFFT: n2=3.30e-07 ninf=8.94e-07 < 8.87e-06 (0.101) 0   OK
pyopencl  R2C    (274,274+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=1.01e-06 ninf=1.04e-06 < 8.88e-06 (0.117) 0 iFFT: n2=1.18e+01 ninf=4.58e+01 < 8.88e-06 (5157324.776) 0 FAIL
pyopencl  R2C    (275,275+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.34e-07 ninf=6.30e-07 < 8.88e-06 (0.071) 0 iFFT: n2=3.79e-07 ninf=9.54e-07 < 8.88e-06 (0.107) 0   OK
pyopencl  R2C    (276,276+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.40e-07 ninf=2.30e-07 < 8.88e-06 (0.026) 0 iFFT: n2=2.42e-07 ninf=6.56e-07 < 8.88e-06 (0.074) 0   OK
pyopencl  R2C    (277,277+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=9.93e-07 ninf=1.13e-06 < 8.88e-06 (0.127) 0 iFFT: n2=1.19e+01 ninf=4.24e+01 < 8.88e-06 (4773055.106) 0 FAIL
pyopencl  R2C    (278,278+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=9.98e-07 ninf=9.51e-07 < 4.44e-06 (0.214) 0 iFFT: n2=1.18e+01 ninf=4.18e+01 < 4.44e-06 (9399569.965) 0 FAIL
pyopencl  R2C    (279,279+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.91e-07 ninf=3.36e-07 < 4.45e-06 (0.075) 0 iFFT: n2=2.64e-07 ninf=7.75e-07 < 4.45e-06 (0.174) 0   OK
pyopencl  R2C    (280,280+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.81e-07 ninf=5.93e-07 < 8.89e-06 (0.067) 0 iFFT: n2=4.19e-07 ninf=1.13e-06 < 8.89e-06 (0.127) 0   OK
pyopencl  R2C    (281,281+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.27e-07 ninf=4.73e-07 < 8.90e-06 (0.053) 0 iFFT: n2=3.79e-07 ninf=1.13e-06 < 8.90e-06 (0.127) 0   OK
pyopencl  R2C    (282,282+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=9.72e-07 ninf=9.72e-07 < 8.90e-06 (0.109) 0 iFFT: n2=1.19e+01 ninf=4.08e+01 < 8.90e-06 (4585162.027) 0 FAIL
pyopencl  R2C    (283,283+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=8.35e-07 ninf=1.01e-06 < 8.90e-06 (0.114) 0 iFFT: n2=1.20e+01 ninf=3.92e+01 < 8.90e-06 (4403364.537) 0 FAIL
pyopencl  R2C    (284,284+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.96e-07 ninf=3.19e-07 < 4.45e-06 (0.072) 0 iFFT: n2=2.95e-07 ninf=7.15e-07 < 4.45e-06 (0.161) 0   OK
pyopencl  R2C    (285,285+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.86e-07 ninf=3.54e-07 < 4.45e-06 (0.080) 0 iFFT: n2=2.60e-07 ninf=7.75e-07 < 4.45e-06 (0.174) 0   OK
pyopencl  R2C    (286,286+2) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=4.39e-07 ninf=5.02e-07 < 4.46e-06 (0.113) 0 iFFT: n2=4.26e-07 ninf=1.31e-06 < 4.46e-06 (0.294) 0   OK
pyopencl  R2C    (287,287+1) axes=      None ndim=   2    float32 lut=False inplace=1  norm=   1 C   FFT: n2=2.57e-07 ninf=2.86e-07 < 4.46e-06 (0.064) 0 iFFT: n2=2.36e-07 ninf=5.96e-07 < 4.46e-06 (0.134) 0   OK
...

... and the same test starts failing with n=653, 659... on an nvidia A2000 - using opencl. Same card with cuda fails starting at n=1543,1545,1546..

Dear @vincefn

All the mentioned issues should be fixed. Also, you don't need to set forceCallbackVersionRealTransforms yourself, it will be automatically enabled for multi-upload odd algorithm. For single upload cases, the existing shared memory algorithm can save up to 2x floating point operations for multidimensional FFTs.

Thanks ! I removed the forceCallbackVersionRealTransforms for odd-sized r2c, all good. Now all the C2C, R2C, DCT tests pass.

There are some remaining issues with DST1 inverse transforms for some sizes (radix or not), e.g. in 1, 2 and 3D - onset is much larger in 2D:

pyvkfft-test --nproc 4 --systematic --dst 1 --range 2 1000 | grep FAIL

pyopencl DST1          (46,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.30e-07 ninf=2.47e-07 < 5.66e-06 (0.044) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.66e-06 (353184.759) 1 FAIL
pyopencl DST1          (58,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.08e-07 ninf=4.70e-07 < 5.76e-06 (0.082) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.76e-06 (347015.638) 1 FAIL
pyopencl DST1          (82,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.10e-07 ninf=2.41e-07 < 5.91e-06 (0.041) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.91e-06 (338191.233) 1 FAIL
pyopencl DST1          (93,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.46e-07 ninf=4.32e-07 < 5.97e-06 (0.072) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.97e-06 (335093.507) 1 FAIL
pyopencl DST1         (102,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.59e-07 ninf=4.03e-07 < 6.01e-06 (0.067) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.01e-06 (332856.170) 1 FAIL
pyopencl DST1         (106,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.96e-07 ninf=2.64e-07 < 6.03e-06 (0.044) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.03e-06 (331933.357) 1 FAIL
pyopencl DST1         (117,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.43e-07 ninf=4.89e-07 < 3.03e-06 (0.161) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 3.03e-06 (659175.473) 1 FAIL
pyopencl DST1         (136,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.83e-07 ninf=6.49e-07 < 6.13e-06 (0.106) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.13e-06 (326076.008) 1 FAIL
pyopencl DST1         (138,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.83e-07 ninf=5.05e-07 < 6.14e-06 (0.082) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.14e-06 (325739.217) 1 FAIL
pyopencl DST1         (140,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.95e-07 ninf=5.67e-07 < 3.07e-06 (0.185) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 3.07e-06 (650816.211) 1 FAIL
pyopencl DST1         (148,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.72e-07 ninf=3.82e-07 < 6.17e-06 (0.062) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.17e-06 (324135.302) 1 FAIL
pyopencl DST1         (165,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.83e-07 ninf=4.19e-07 < 3.11e-06 (0.135) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 3.11e-06 (643346.942) 1 FAIL
pyopencl DST1         (166,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.89e-07 ninf=5.04e-07 < 6.22e-06 (0.081) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.22e-06 (321537.783) 1 FAIL
pyopencl DST1         (172,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.85e-07 ninf=4.82e-07 < 6.24e-06 (0.077) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.24e-06 (320742.644) 1 FAIL
pyopencl DST1         (176,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=5.10e-07 ninf=5.08e-07 < 3.12e-06 (0.163) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 3.12e-06 (640459.642) 1 FAIL
pyopencl DST1         (178,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.89e-07 ninf=5.95e-07 < 6.25e-06 (0.095) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.25e-06 (319978.402) 1 FAIL
pyopencl DST1         (187,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=5.05e-07 ninf=5.63e-07 < 6.27e-06 (0.090) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.27e-06 (318885.528) 1 FAIL
pyopencl DST1         (190,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=5.27e-07 ninf=6.19e-07 < 6.28e-06 (0.099) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.28e-06 (318534.501) 1 FAIL
pyopencl DST1         (205,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.83e-07 ninf=4.22e-07 < 6.31e-06 (0.067) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.31e-06 (316869.081) 1 FAIL
pyopencl DST1         (213,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.07e-07 ninf=4.46e-07 < 6.33e-06 (0.071) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.33e-06 (316036.611) 1 FAIL
pyopencl DST1         (222,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.22e-07 ninf=5.16e-07 < 6.35e-06 (0.081) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.35e-06 (315141.570) 1 FAIL
pyopencl DST1         (226,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.84e-07 ninf=3.70e-07 < 6.35e-06 (0.058) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.35e-06 (314756.926) 1 FAIL
pyopencl DST1         (228,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.14e-07 ninf=4.95e-07 < 6.36e-06 (0.078) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.36e-06 (314567.477) 1 FAIL
pyopencl DST1         (232,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.75e-07 ninf=5.04e-07 < 6.37e-06 (0.079) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.37e-06 (314194.256) 1 FAIL
pyopencl DST1         (234,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.93e-07 ninf=5.09e-07 < 3.18e-06 (0.160) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 3.18e-06 (628020.719) 1 FAIL
pyopencl DST1         (235,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.02e-07 ninf=4.80e-07 < 6.37e-06 (0.075) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.37e-06 (313919.061) 1 FAIL
pyopencl DST1         (238,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.09e-07 ninf=4.83e-07 < 6.38e-06 (0.076) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 6.38e-06 (313647.811) 1 FAIL
...
pyvkfft-test --nproc 4 --systematic --dst 1 --range 2 1000 --ndim 2 | grep FAIL

pyopencl DST1      (520,520) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.92e-07 ninf=5.44e-07 < 4.72e-06 (0.115) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.72e-06 (424087.859) 1 FAIL
pyopencl DST1      (540,540) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.41e-07 ninf=4.23e-07 < 4.73e-06 (0.089) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.73e-06 (422619.077) 1 FAIL
pyopencl DST1      (546,546) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.79e-07 ninf=4.40e-07 < 4.74e-06 (0.093) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.74e-06 (422190.929) 1 FAIL
pyopencl DST1      (576,576) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.86e-07 ninf=4.21e-07 < 4.76e-06 (0.088) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.76e-06 (420130.703) 1 FAIL
pyopencl DST1      (600,600) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.51e-07 ninf=4.27e-07 < 4.78e-06 (0.089) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.78e-06 (418571.857) 1 FAIL
pyopencl DST1      (616,616) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.69e-07 ninf=4.88e-07 < 4.79e-06 (0.102) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.79e-06 (417573.012) 1 FAIL
pyopencl DST1      (630,630) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.10e-07 ninf=4.32e-07 < 4.80e-06 (0.090) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.80e-06 (416723.819) 1 FAIL
pyopencl DST1      (640,640) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.31e-07 ninf=4.35e-07 < 4.81e-06 (0.091) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.81e-06 (416130.801) 1 FAIL
pyopencl DST1      (660,660) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.82e-07 ninf=4.09e-07 < 4.82e-06 (0.085) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.82e-06 (414976.975) 1 FAIL
pyopencl DST1      (672,672) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.90e-07 ninf=6.17e-07 < 4.83e-06 (0.128) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.83e-06 (414304.282) 1 FAIL
pyopencl DST1      (676,676) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.96e-07 ninf=4.76e-07 < 4.83e-06 (0.099) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.83e-06 (414083.195) 1 FAIL
pyopencl DST1      (700,700) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.14e-07 ninf=4.97e-07 < 4.85e-06 (0.103) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.85e-06 (412788.296) 1 FAIL
pyopencl DST1      (726,726) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.74e-07 ninf=3.92e-07 < 4.86e-06 (0.081) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.86e-06 (411443.267) 1 FAIL
pyopencl DST1      (750,750) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.77e-07 ninf=4.08e-07 < 4.88e-06 (0.084) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.88e-06 (410251.255) 1 FAIL
pyopencl DST1      (756,756) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.25e-07 ninf=4.91e-07 < 4.88e-06 (0.101) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.88e-06 (409960.247) 1 FAIL
pyopencl DST1      (768,768) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.62e-07 ninf=5.32e-07 < 4.89e-06 (0.109) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.89e-06 (409386.309) 1 FAIL
pyopencl DST1      (810,810) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.46e-07 ninf=4.67e-07 < 4.91e-06 (0.095) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.91e-06 (407457.672) 1 FAIL
pyopencl DST1      (858,858) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.04e-07 ninf=4.54e-07 < 4.93e-06 (0.092) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.93e-06 (405392.754) 1 FAIL
pyopencl DST1      (880,880) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.89e-07 ninf=5.06e-07 < 4.94e-06 (0.102) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.94e-06 (404491.255) 1 FAIL
pyopencl DST1      (882,882) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.66e-07 ninf=4.81e-07 < 4.95e-06 (0.097) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.95e-06 (404410.568) 1 FAIL
pyopencl DST1      (910,910) axes=      None ndim=   2    float32 lut=False inplace=0  norm=   1 C   FFT: n2=4.29e-07 ninf=4.95e-07 < 4.96e-06 (0.100) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 4.96e-06 (403303.800) 1 FAIL
...

pyvkfft-test --nproc 4 --systematic --dst 1 --range 2 200 --ndim 3 | grep FAIL

pyopencl DST1     (46,46,46) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.77e-07 ninf=7.98e-07 < 8.99e-06 (0.089) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 8.99e-06 (222512.064) 1 FAIL
pyopencl DST1     (58,58,58) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=7.07e-07 ninf=7.97e-07 < 9.29e-06 (0.086) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 9.29e-06 (215278.595) 1 FAIL
pyopencl DST1     (82,82,82) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=9.10e-07 ninf=1.10e-06 < 9.74e-06 (0.113) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 9.74e-06 (205308.325) 1 FAIL
pyopencl DST1     (93,93,93) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=9.26e-07 ninf=9.71e-07 < 9.91e-06 (0.098) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 9.91e-06 (201908.953) 1 FAIL
pyopencl DST1  (102,102,102) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=9.51e-07 ninf=1.03e-06 < 1.00e-05 (0.102) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 1.00e-05 (199485.222) 1 FAIL
pyopencl DST1  (106,106,106) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=9.42e-07 ninf=1.06e-06 < 1.01e-05 (0.105) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 1.01e-05 (198492.970) 1 FAIL
pyopencl DST1  (117,117,117) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=9.77e-07 ninf=1.05e-06 < 5.10e-06 (0.207) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.10e-06 (391981.506) 1 FAIL
pyopencl DST1  (136,136,136) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.39e-06 ninf=1.40e-06 < 1.04e-05 (0.135) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 1.04e-05 (192296.174) 1 FAIL
pyopencl DST1  (138,138,138) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.41e-06 ninf=1.51e-06 < 1.04e-05 (0.145) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 1.04e-05 (191945.136) 1 FAIL
pyopencl DST1  (140,140,140) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.38e-06 ninf=1.46e-06 < 5.22e-06 (0.280) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 5.22e-06 (383200.846) 1 FAIL
pyopencl DST1  (148,148,148) axes=      None ndim=   3    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.36e-06 ninf=1.56e-06 < 1.05e-05 (0.149) 1 iFFT: n2=2.00e+00 ninf=2.00e+00 < 1.05e-05 (190280.606) 1 FAIL

Sorry, missed one other place where I had to apply DST-I change. Should be fixed now (my pyvkfft version is still running with no fails where it failed before fix).

Ok, looks good.

While testing this I noticed something strange with 2D R2C transforms where the second dimension (ny) is odd (well, I just tested with ny=511 to be honest) - the maximum error is larger than expected, only for the last row, as you can see:

Nothing huge - but still about 1 order of magnitude larger than what you have for the remaining of the array. Is that normal ?

I have not tested other transforms with this (c2c, r2r,, double precision etc..) - in my test suite I mostly test square dimensions, though in the long version there are asymmetric sizes testing..

Hmm, this may be related to the merging algorithm that uses the same system twice for the last row. I wonder if replacing vkFFT\vkFFT_CodeGen\vkFFT_KernelsLevel1\vkFFT_ReadWrite with the following archive contents solves it.

vkFFT_ReadWrite.zip

Not better:

Here is some self-contained code to reproduce the issue:

# Uncomment to have interactive plots (with jupyterlab)
%matplotlib widget

import pyopencl as cl
import pyopencl.array as cla
from pyvkfft.fft import rfftn as vkrfftn, irfftn as vkirfftn
from pyvkfft.opencl import VkFFTApp
import matplotlib.pyplot as plt
try:
    from scipy.datasets import ascent
except:
    from scipy.misc import ascent
import numpy as np

ctx = None
# Find the first OpenCL GPU available and use it, unless
for p in cl.get_platforms():
    for d in p.get_devices():
        if d.type & cl.device_type.GPU == 0:
            continue
        print("Selected device: ", d.name)
        ctx = cl.Context(devices=(d,))
        break
    if ctx is not None:
        break
cq = cl.CommandQueue(ctx)

ny = 511
nx = 512
d0 = ascent()[:ny,:nx].astype(np.float32)
d = cla.to_device(cq, d0)
d = vkirfftn(vkrfftn(d), r2c_odd=(nx%2))  # Also handle the case where nx is odd

plt.figure(figsize=(15,4))
plt.subplot(141)
plt.imshow(d0)
plt.title(f'original ny={ny} nx={nx}')
plt.subplot(142)
plt.imshow(d.get())
plt.title('R2C+C2R (out-of-place)')
plt.subplot(143)
plt.imshow(d0-d.get())
plt.ylim(ny,ny-10)
plt.gca().set_aspect('auto')
plt.title('difference')
plt.subplot(144)
plt.plot(abs(d0-d.get()).mean(axis=-1),'.-')
plt.title('$<|d0-d|>_x(y)$')
plt.xlabel('iy')
plt.tight_layout()

To look more systematically at this, here are the plots for the long test suite (I ran it without the patch) for an R2C out-of-place transform, 2D, single precision, cuda, A40, non-radix: Result from July - only on even sizes:

Result from the current version - same config but now also includes the odd sizes:

This suggests that the accuracy is significantly worse for odd-sized R2C. As you will see below, this creates accuracy failures in the long tests.

You can re-run the accuracy test for all sizes (radix and non-radix) using e.g. : pyvkfft-test --systematic --backend pycuda --gpu a40 --max-nb-tests 0 --nproc 16 --ndim 2 --range 2 4500 --r2c --norm 1 --range-mb 0 4100

For some reason the first failure occurs at n=849 using --inplace, but only at 1759 for an out-of-place transform. That's about twice the size, so that may give you a clue as to what happens.

Full test results

When looking at the full test suite, there are a number of issues detected:

for 2D and 3D R2C inplace transforms, there are a number of accuracy failures - in this case the maximum error (L_inf) is too large for the inverse transform, but only for larger sizes. That should be related to the above. Click on the coloured 'R2C' to see the accuracy plot, and click on the number of failures to see the details, which seem to indicate all failures are for odd sizes.
there are a number of failures and errors (launching or memory errors) for DCT/DST transforms. It is worse for opencl than cuda

Here are the tests:

Another specific case: 1D R2C out-of-place (inplace were not tested) transforms on non-radix sizes seem to fail both for forward transforms only for odd sizes between 5489 to 5879, both for float32 (only opencl) and float64 (opencl and cuda).

On a more positive note, all the C2C tests pass, as do all the even R2C :-)

@vincefn the accuracy drop for odd batching of C2R should be fixed (previously it used 0,0 element as the imaginary part for the last row, which was a really bad choice, as after R2C it is magnitudes bigger than all other elements). I will investigate other failures now.

@vincefn 5489 to 5879 systems should also be fixed - I was not setting zero-padded values to zero in multi-upload strided reads for R2C. I am trying to isolate the singular failing DCT and DST systems.

@vincefn all the fails and errors for both cuda and opencl should be fixed now. I am running tests that failed before locally, but they seem to pass so far.

Great - the tests are ongoing and look much better:

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-11-29-a40cl/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-11-29-a40cu/pyvkfft-test.html

Note that among the errors from large 3D transforms, it could be due to the parallel execution - for example if I re-run the 3D DCT1 in double precision with --nproc 4 instead of --nproc 11 it passes. I don't know why, the 550**3 data array in flot64 only takes 1.2 GB, so even with 2x11 copies it should still fit without issue in the 48GB of the A40..

Maybe this is rather due to an error with the delay to launch the kernel - or some other saturation of opencl resources..

However there are other errors, some pycuda DST1 issues with size 2048, and other memory issues for DST2&3 2D transforms with sizes 3718, 3960 etc..

@vincefn the DCT-I is solved as 2N-1 system which (if it is Bluestein's algorithm) can be 4N in size and has to be done out-of-place for strided accesses cause original buffer is not sufficient for intermediate storage increasing the memory required 4.5x (roughly). For CUDA, it is likely that more systems fit in single-upload variant and that memory requirement doesn't manifest into an error.

I will check and fix the remaining errors later today, thanks for pointing them out!

OK, the tests are taking some time - now much longer for DCT/DST with longer sizes allowed.

For the CUDA version, the non-radix (Bluestein or radix, I don't know) have a number of cuda memory errors for DCT4 & DST2,3,4 - maybe related to the same errors on radix sizes with specific lengths ?

@vincefn I have finally fixed the issues. While some of them were legit, the issues with DST-II resulted in the solution that looks like this:

if (inoutID_y < 3718) {
stageInvocationID = 3717 - inoutID_y;
if (inoutID_y == 3717) {
stageInvocationID = 0;
}
}

It doesn't make sense, but works. I hope all these fixes solve the remaining fails (the tests on my machine with 3070 pass so far).

Strange fix indeed !

Now, a marathon of tests has started: http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-a40cu/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-a40cl/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-gtx1080ticu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-a100cu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-v100cu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-h100cu/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-h100cl/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-apple-m1/pyvkfft-test.html <- errors

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-01-amdgfx900/pyvkfft-test.html (this last one-AMD gfx900 test has crashed again (not showing the log) - that driver is really unreliable.., I'll try to re-launch it)

So far on the nvidia hosts, a few errors on the A40-OpenCL, but these are just 3D errors likely due to too much memory used (though I reduced the parallelism) - can be ignored.

The Apple M1 tests have some failures on DCT4/DST4 (n=2988 - not that the F or C-order seems to matter), DCT1/DST1 (n>=98010) and R2C (n=13000, batched):

pyopencl DCT4       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 F   FFT: n2=2.52e-01 ninf=2.60e-01 < 7.48e-06 (34807.347) 0 iFFT: n2=2.52e-01 ninf=2.60e-01 < 7.48e-06 (34807.345) 0 FAIL
pyopencl DCT4       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 F   FFT: n2=2.49e-01 ninf=2.08e-01 < 7.48e-06 (27856.546) 1 iFFT: n2=2.49e-01 ninf=2.08e-01 < 7.48e-06 (27856.538) 1 FAIL
pyopencl DCT4       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 F   FFT: n2=2.43e-01 ninf=2.49e-01 < 7.48e-06 (33310.094) 0 iFFT: n2=2.43e-01 ninf=2.49e-01 < 7.48e-06 (33310.090) 0 FAIL
pyopencl DCT4       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 F   FFT: n2=2.54e-01 ninf=2.61e-01 < 7.48e-06 (34914.268) 1 iFFT: n2=2.54e-01 ninf=2.61e-01 < 7.48e-06 (34914.266) 1 FAIL
pyopencl DCT4       (2988,2) axes=      [-2] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.45e-01 ninf=2.46e-01 < 7.48e-06 (32953.123) 0 iFFT: n2=2.45e-01 ninf=2.46e-01 < 7.48e-06 (32953.120) 0 FAIL
pyopencl DCT4       (2988,2) axes=      [-2] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.42e-01 ninf=2.47e-01 < 7.48e-06 (33081.788) 1 iFFT: n2=2.42e-01 ninf=2.47e-01 < 7.48e-06 (33081.785) 1 FAIL
pyopencl DCT4       (2988,2) axes=      [-2] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.52e-01 ninf=2.49e-01 < 7.48e-06 (33334.306) 0 iFFT: n2=2.52e-01 ninf=2.49e-01 < 7.48e-06 (33334.306) 0 FAIL
pyopencl DCT4       (2988,2) axes=      [-2] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.52e-01 ninf=2.52e-01 < 7.48e-06 (33721.976) 1 iFFT: n2=2.52e-01 ninf=2.52e-01 < 7.48e-06 (33721.970) 1 FAIL
pyopencl DCT4       (2,2988) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 F   FFT: n2=2.47e-01 ninf=2.35e-01 < 7.48e-06 (31375.922) 0 iFFT: n2=2.47e-01 ninf=2.35e-01 < 7.48e-06 (31375.920) 0 FAIL
pyopencl DCT4       (2,2988) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 F   FFT: n2=2.42e-01 ninf=2.59e-01 < 7.48e-06 (34691.956) 1 iFFT: n2=2.42e-01 ninf=2.59e-01 < 7.48e-06 (34691.949) 1 FAIL
pyopencl DCT4       (2,2988) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 F   FFT: n2=2.46e-01 ninf=2.55e-01 < 7.48e-06 (34159.304) 0 iFFT: n2=2.46e-01 ninf=2.55e-01 < 7.48e-06 (34159.299) 0 FAIL
pyopencl DCT4       (2,2988) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 F   FFT: n2=2.46e-01 ninf=2.11e-01 < 7.48e-06 (28218.334) 1 iFFT: n2=2.46e-01 ninf=2.11e-01 < 7.48e-06 (28218.335) 1 FAIL
pyopencl DST4       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 F   FFT: n2=2.44e-01 ninf=2.97e-01 < 7.48e-06 (39717.428) 0 iFFT: n2=2.44e-01 ninf=2.97e-01 < 7.48e-06 (39717.423) 0 FAIL
pyopencl DST4       (2,2988) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 F   FFT: n2=2.48e-01 ninf=2.42e-01 < 7.48e-06 (32382.720) 1 iFFT: n2=2.48e-01 ninf=2.42e-01 < 7.48e-06 (32382.717) 1 FAIL
pyopencl DST4       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 F   FFT: n2=2.43e-01 ninf=2.46e-01 < 7.48e-06 (32913.538) 0 iFFT: n2=2.43e-01 ninf=2.46e-01 < 7.48e-06 (32913.547) 0 FAIL
pyopencl DST4       (2,2988) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 F   FFT: n2=2.46e-01 ninf=2.49e-01 < 7.48e-06 (33311.601) 1 iFFT: n2=2.46e-01 ninf=2.49e-01 < 7.48e-06 (33311.604) 1 FAIL
pyopencl DST4       (2988,2) axes=      [-2] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=2.47e-01 ninf=2.39e-01 < 7.48e-06 (31983.367) 0 iFFT: n2=2.47e-01 ninf=2.39e-01 < 7.48e-06 (31983.372) 0 FAIL
pyopencl DST4       (2988,2) axes=      [-2] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=2.48e-01 ninf=2.12e-01 < 7.48e-06 (28341.393) 1 iFFT: n2=2.48e-01 ninf=2.12e-01 < 7.48e-06 (28341.391) 1 FAIL
pyopencl DST4       (2988,2) axes=      [-2] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=2.48e-01 ninf=2.44e-01 < 7.48e-06 (32678.782) 0 iFFT: n2=2.48e-01 ninf=2.44e-01 < 7.48e-06 (32678.782) 0 FAIL
pyopencl DST4       (2988,2) axes=      [-2] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=2.55e-01 ninf=2.84e-01 < 7.48e-06 (38044.469) 1 iFFT: n2=2.55e-01 ninf=2.84e-01 < 7.48e-06 (38044.460) 1 FAIL
pyopencl DST4       (2,2988) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 F   FFT: n2=2.43e-01 ninf=2.60e-01 < 7.48e-06 (34779.685) 0 iFFT: n2=2.43e-01 ninf=2.60e-01 < 7.48e-06 (34779.684) 0 FAIL
pyopencl DST4       (2,2988) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 F   FFT: n2=2.49e-01 ninf=2.52e-01 < 7.48e-06 (33724.834) 1 iFFT: n2=2.49e-01 ninf=2.52e-01 < 7.48e-06 (33724.835) 1 FAIL
pyopencl DST4       (2,2988) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 F   FFT: n2=2.41e-01 ninf=2.59e-01 < 7.48e-06 (34603.130) 0 iFFT: n2=2.41e-01 ninf=2.59e-01 < 7.48e-06 (34603.129) 0 FAIL
pyopencl DST4       (2,2988) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 F   FFT: n2=2.47e-01 ninf=2.54e-01 < 7.48e-06 (34019.848) 1 iFFT: n2=2.47e-01 ninf=2.54e-01 < 7.48e-06 (34019.836) 1 FAIL
[...]
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.37e-07 ninf=4.01e-07 < 8.11e-06 (0.049) 0 iFFT: n2=1.96e-01 ninf=5.83e-01 < 8.11e-06 (71805.472) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.36e-07 ninf=3.59e-07 < 4.06e-06 (0.088) 1 iFFT: n2=1.73e-01 ninf=5.49e-01 < 4.06e-06 (135324.477) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.55e-07 ninf=1.68e-07 < 4.06e-06 (0.042) 1 iFFT: n2=4.77e-02 ninf=1.09e-01 < 4.06e-06 (26941.339) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.37e-07 ninf=4.08e-07 < 8.11e-06 (0.050) 0 iFFT: n2=9.03e-02 ninf=2.15e-01 < 8.11e-06 (26531.328) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.37e-07 ninf=4.14e-07 < 8.11e-06 (0.051) 0 iFFT: n2=1.44e-01 ninf=4.18e-01 < 8.11e-06 (51563.642) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=4.04e-07 < 4.06e-06 (0.100) 1 iFFT: n2=8.48e-02 ninf=2.32e-01 < 4.06e-06 (57173.935) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.76e-07 < 8.11e-06 (0.022) 0 iFFT: n2=7.09e-01 ninf=3.06e+00 < 8.11e-06 (377448.887) 0 FAIL
[...]
pyopencl DCT1       (98010,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.12e+00 < 4.50e-06 (249359.056) 1 iFFT: n2=1.15e+00 ninf=1.12e+00 < 4.50e-06 (249359.083) 1 FAIL
pyopencl DCT1       (98098,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.14e+00 ninf=1.15e+00 < 4.50e-06 (256055.190) 1 iFFT: n2=1.14e+00 ninf=1.15e+00 < 4.50e-06 (256055.190) 1 FAIL
pyopencl DCT1       (98280,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.12e+00 < 4.50e-06 (249949.012) 1 iFFT: n2=1.15e+00 ninf=1.12e+00 < 4.50e-06 (249949.012) 1 FAIL
pyopencl DCT1       (98304,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.10e+00 < 4.50e-06 (244846.877) 1 iFFT: n2=1.15e+00 ninf=1.10e+00 < 4.50e-06 (244846.877) 1 FAIL
pyopencl DCT1       (98415,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.37e+00 < 4.50e-06 (304049.565) 1 iFFT: n2=1.15e+00 ninf=1.37e+00 < 4.50e-06 (304049.565) 1 FAIL
pyopencl DCT1       (98560,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.08e+00 < 4.50e-06 (240070.103) 1 iFFT: n2=1.15e+00 ninf=1.08e+00 < 4.50e-06 (240070.103) 1 FAIL
pyopencl DCT1       (98784,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.14e+00 < 4.50e-06 (252604.949) 1 iFFT: n2=1.15e+00 ninf=1.14e+00 < 4.50e-06 (252604.949) 1 FAIL
pyopencl DCT1       (98865,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.35e+00 < 4.50e-06 (299684.666) 1 iFFT: n2=1.15e+00 ninf=1.35e+00 < 4.50e-06 (299684.692) 1 FAIL
pyopencl DCT1       (99000,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.16e+00 ninf=1.27e+00 < 4.50e-06 (282501.110) 1 iFFT: n2=1.16e+00 ninf=1.27e+00 < 4.50e-06 (282501.110) 1 FAIL
pyopencl DCT1       (99099,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.16e+00 < 4.50e-06 (257446.910) 1 iFFT: n2=1.15e+00 ninf=1.16e+00 < 4.50e-06 (257446.910) 1 FAIL
pyopencl DCT1       (99372,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.16e+00 ninf=1.30e+00 < 4.50e-06 (288691.411) 1 iFFT: n2=1.16e+00 ninf=1.30e+00 < 4.50e-06 (288691.411) 1 FAIL
pyopencl DCT1       (99792,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.00e+00 < 4.50e-06 (222761.072) 1 iFFT: n2=1.15e+00 ninf=1.00e+00 < 4.50e-06 (222761.072) 1 FAIL
pyopencl DCT1       (99825,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.18e+00 < 4.50e-06 (261935.889) 1 iFFT: n2=1.15e+00 ninf=1.18e+00 < 4.50e-06 (261935.889) 1 FAIL
pyopencl DCT1       (99840,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.15e+00 ninf=1.15e+00 < 4.50e-06 (255114.797) 1 iFFT: n2=1.15e+00 ninf=1.15e+00 < 4.50e-06 (255114.797) 1 FAIL

However note that strangely, I cannot reproduce the R2C errors on my macbook pro M1 (the test suite linked above is running on a mac mini, slightly different chip).

There are a number of other errors coming up for DCT4/DST4 on the nvidia cards (A40, A100, H100) for Bluestein (or Rader, I don't know how to determine which since it can depend on the GPU).

A few 3D non-radix DCT2/3 are also failing on the M1.

@vincefn I have fixed (98010,) etc issues for M1, but the rest seems to be tougher.

I emulated M1 parameters on Nvidia GPU so that both GPUs run the same generated code and I couldn't reproduce 4098 DCT-IV error. I will need more time to fix all the other issues.

Also, 13000 R2C issues also were not present on my M1 MacBook.

I haven't checked DCT4/DST4 errors on the Nvidia cards yet, but thanks for pointing them out.

It does seem that the code is getting too complex for some cases that there may be compiler issues that are hard to fix.

All Nvidia issues should be fixed. I will investigate M1 issues now.

Thanks- the M1/r2c/13000 issues are really strange, only occuring on the mac mini M1 (and not a macbook pro M1), and only for some batched transforms configurations, even varying with the norm:

pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.37e-07 ninf=4.47e-07 < 8.11e-06 (0.055) 0 iFFT: n2=3.23e-07 ninf=7.67e-07 < 8.11e-06 (0.095) 0   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.39e-07 ninf=3.80e-07 < 8.11e-06 (0.047) 0 iFFT: n2=3.67e-07 ninf=7.81e-07 < 8.11e-06 (0.096) 0   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.36e-07 ninf=3.81e-07 < 4.06e-06 (0.094) 1 iFFT: n2=3.23e-07 ninf=9.57e-07 < 4.06e-06 (0.236) 1   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.36e-07 ninf=3.70e-07 < 4.06e-06 (0.091) 1 iFFT: n2=3.71e-07 ninf=8.82e-07 < 4.06e-06 (0.217) 1   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.55e-07 ninf=2.09e-07 < 8.11e-06 (0.026) 0 iFFT: n2=1.53e-07 ninf=3.62e-07 < 8.11e-06 (0.045) 0   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.70e-07 < 8.11e-06 (0.021) 0 iFFT: n2=1.60e-07 ninf=3.79e-07 < 8.11e-06 (0.047) 0   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.55e-07 ninf=1.53e-07 < 4.06e-06 (0.038) 1 iFFT: n2=1.53e-07 ninf=4.12e-07 < 4.06e-06 (0.102) 1   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.56e-07 ninf=1.74e-07 < 4.06e-06 (0.043) 1 iFFT: n2=1.62e-07 ninf=4.07e-07 < 4.06e-06 (0.100) 1   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.37e-07 ninf=4.03e-07 < 8.11e-06 (0.050) 0 iFFT: n2=3.23e-07 ninf=8.44e-07 < 8.11e-06 (0.104) 0   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.39e-07 ninf=3.71e-07 < 8.11e-06 (0.046) 0 iFFT: n2=3.68e-07 ninf=8.41e-07 < 8.11e-06 (0.104) 0   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.38e-07 ninf=4.72e-07 < 4.06e-06 (0.116) 1 iFFT: n2=3.20e-07 ninf=7.77e-07 < 4.06e-06 (0.191) 1   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=4.19e-07 < 4.06e-06 (0.103) 1 iFFT: n2=3.66e-07 ninf=8.77e-07 < 4.06e-06 (0.216) 1   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.84e-07 < 8.11e-06 (0.023) 0 iFFT: n2=1.53e-07 ninf=4.31e-07 < 8.11e-06 (0.053) 0   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.56e-07 < 8.11e-06 (0.019) 0 iFFT: n2=1.62e-07 ninf=4.70e-07 < 8.11e-06 (0.058) 0   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=1.66e-07 < 4.06e-06 (0.041) 1 iFFT: n2=1.53e-07 ninf=3.54e-07 < 4.06e-06 (0.087) 1   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.55e-07 ninf=1.62e-07 < 4.06e-06 (0.040) 1 iFFT: n2=1.62e-07 ninf=4.21e-07 < 4.06e-06 (0.104) 1   OK
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.36e-07 ninf=3.52e-07 < 8.11e-06 (0.043) 0 iFFT: n2=3.23e-07 ninf=9.07e-07 < 8.11e-06 (0.112) 0   OK
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.35e-07 ninf=3.74e-07 < 8.11e-06 (0.046) 0 iFFT: n2=1.22e-01 ninf=3.37e-01 < 8.11e-06 (41511.824) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.38e-07 ninf=3.56e-07 < 4.06e-06 (0.088) 1 iFFT: n2=1.91e-01 ninf=5.60e-01 < 4.06e-06 (138083.065) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=3.85e-07 < 4.06e-06 (0.095) 1 iFFT: n2=9.93e-02 ninf=2.70e-01 < 4.06e-06 (66671.769) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=2.14e-07 < 8.11e-06 (0.026) 0 iFFT: n2=1.54e-07 ninf=4.17e-07 < 8.11e-06 (0.051) 0   OK
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.46e-07 < 8.11e-06 (0.018) 0 iFFT: n2=1.61e-07 ninf=3.91e-07 < 8.11e-06 (0.048) 0   OK
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=2.00e-07 < 4.06e-06 (0.049) 1 iFFT: n2=7.17e-02 ninf=2.03e-01 < 4.06e-06 (50041.149) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.55e-07 ninf=1.84e-07 < 4.06e-06 (0.045) 1 iFFT: n2=1.61e-07 ninf=4.12e-07 < 4.06e-06 (0.102) 1   OK
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.35e-07 ninf=4.12e-07 < 8.11e-06 (0.051) 0 iFFT: n2=1.22e-01 ninf=4.11e-01 < 8.11e-06 (50617.132) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.37e-07 ninf=4.26e-07 < 8.11e-06 (0.053) 0 iFFT: n2=7.66e-02 ninf=2.00e-01 < 8.11e-06 (24692.495) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.38e-07 ninf=3.72e-07 < 4.06e-06 (0.092) 1 iFFT: n2=1.05e-01 ninf=2.81e-01 < 4.06e-06 (69227.634) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=3.98e-07 < 4.06e-06 (0.098) 1 iFFT: n2=1.18e-01 ninf=3.80e-01 < 4.06e-06 (93736.032) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.67e-07 < 8.11e-06 (0.021) 0 iFFT: n2=9.43e-02 ninf=2.47e-01 < 8.11e-06 (30450.235) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.76e-07 < 8.11e-06 (0.022) 0 iFFT: n2=1.60e-07 ninf=4.27e-07 < 8.11e-06 (0.053) 0   OK
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=1.77e-07 < 4.06e-06 (0.044) 1 iFFT: n2=1.53e-07 ninf=4.28e-07 < 4.06e-06 (0.105) 1   OK
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.54e-07 ninf=1.90e-07 < 4.06e-06 (0.047) 1 iFFT: n2=1.61e-07 ninf=3.97e-07 < 4.06e-06 (0.098) 1   OK

What is also strange is that a transform of an array of shape (2,2,13000) is not different from one on an array of size (2,13000) - the batch size just changes.

Out of curiosity I tried this again with --serial:

pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.37e-07 ninf=3.97e-07 < 8.11e-06 (0.049) 0 iFFT: n2=3.24e-07 ninf=7.83e-07 < 8.11e-06 (0.097) 0   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.37e-07 ninf=3.97e-07 < 8.11e-06 (0.049) 0 iFFT: n2=3.68e-07 ninf=7.86e-07 < 8.11e-06 (0.097) 0   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.37e-07 ninf=3.97e-07 < 4.06e-06 (0.098) 1 iFFT: n2=3.24e-07 ninf=7.83e-07 < 4.06e-06 (0.193) 1   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=3.97e-07 < 4.06e-06 (0.098) 1 iFFT: n2=3.68e-07 ninf=7.86e-07 < 4.06e-06 (0.194) 1   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.73e-07 < 8.11e-06 (0.021) 0 iFFT: n2=1.53e-07 ninf=3.59e-07 < 8.11e-06 (0.044) 0   OK
pyopencl  R2C    (2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.73e-07 < 8.11e-06 (0.021) 0 iFFT: n2=1.62e-07 ninf=4.15e-07 < 8.11e-06 (0.051) 0   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=1.73e-07 < 4.06e-06 (0.043) 1 iFFT: n2=1.53e-07 ninf=3.59e-07 < 4.06e-06 (0.089) 1   OK
pyopencl  R2C      (2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.54e-07 ninf=1.73e-07 < 4.06e-06 (0.043) 1 iFFT: n2=1.62e-07 ninf=4.15e-07 < 4.06e-06 (0.102) 1   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.38e-07 ninf=4.26e-07 < 8.11e-06 (0.052) 0 iFFT: n2=3.22e-07 ninf=8.26e-07 < 8.11e-06 (0.102) 0   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.38e-07 ninf=4.26e-07 < 8.11e-06 (0.052) 0 iFFT: n2=3.67e-07 ninf=1.01e-06 < 8.11e-06 (0.124) 0   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.38e-07 ninf=4.26e-07 < 4.06e-06 (0.105) 1 iFFT: n2=3.22e-07 ninf=8.26e-07 < 4.06e-06 (0.204) 1   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.38e-07 ninf=4.26e-07 < 4.06e-06 (0.105) 1 iFFT: n2=3.67e-07 ninf=1.01e-06 < 4.06e-06 (0.248) 1   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.55e-07 ninf=1.70e-07 < 8.11e-06 (0.021) 0 iFFT: n2=1.53e-07 ninf=3.63e-07 < 8.11e-06 (0.045) 0   OK
pyopencl  R2C    (2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.55e-07 ninf=1.70e-07 < 8.11e-06 (0.021) 0 iFFT: n2=1.60e-07 ninf=3.86e-07 < 8.11e-06 (0.048) 0   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.55e-07 ninf=1.70e-07 < 4.06e-06 (0.042) 1 iFFT: n2=1.53e-07 ninf=3.63e-07 < 4.06e-06 (0.089) 1   OK
pyopencl  R2C      (2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.55e-07 ninf=1.70e-07 < 4.06e-06 (0.042) 1 iFFT: n2=1.60e-07 ninf=3.86e-07 < 4.06e-06 (0.095) 1   OK
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.35e-07 ninf=3.82e-07 < 8.11e-06 (0.047) 0 iFFT: n2=1.24e-01 ninf=3.52e-01 < 8.11e-06 (43423.421) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.35e-07 ninf=3.82e-07 < 8.11e-06 (0.047) 0 iFFT: n2=1.24e-01 ninf=3.52e-01 < 8.11e-06 (43423.414) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.35e-07 ninf=3.82e-07 < 4.06e-06 (0.094) 1 iFFT: n2=1.24e-01 ninf=3.52e-01 < 4.06e-06 (86846.843) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.35e-07 ninf=3.82e-07 < 4.06e-06 (0.094) 1 iFFT: n2=1.24e-01 ninf=3.52e-01 < 4.06e-06 (86846.828) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.60e-07 < 8.11e-06 (0.020) 0 iFFT: n2=1.53e-07 ninf=4.05e-07 < 8.11e-06 (0.050) 0   OK
pyopencl  R2C  (2,2,13000+2) axes=      None ndim=   1    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.60e-07 < 8.11e-06 (0.020) 0 iFFT: n2=1.60e-07 ninf=4.20e-07 < 8.11e-06 (0.052) 0   OK
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=1.60e-07 < 4.06e-06 (0.039) 1 iFFT: n2=1.24e-01 ninf=3.52e-01 < 4.06e-06 (86846.894) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      None ndim=   1    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.54e-07 ninf=1.60e-07 < 4.06e-06 (0.039) 1 iFFT: n2=1.24e-01 ninf=3.52e-01 < 4.06e-06 (86846.865) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   0 C   FFT: n2=3.38e-07 ninf=3.82e-07 < 8.11e-06 (0.047) 0 iFFT: n2=7.46e-02 ninf=2.16e-01 < 8.11e-06 (26665.502) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=None inplace=1  norm=   1 C   FFT: n2=3.38e-07 ninf=3.82e-07 < 8.11e-06 (0.047) 0 iFFT: n2=7.46e-02 ninf=2.16e-01 < 8.11e-06 (26665.502) 0 FAIL
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   0 C   FFT: n2=3.38e-07 ninf=3.82e-07 < 4.06e-06 (0.094) 1 iFFT: n2=7.46e-02 ninf=2.16e-01 < 4.06e-06 (53331.003) 1 FAIL
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=None inplace=0  norm=   1 C   FFT: n2=3.38e-07 ninf=3.82e-07 < 4.06e-06 (0.094) 1 iFFT: n2=7.46e-02 ninf=2.16e-01 < 4.06e-06 (53331.003) 1 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   0 C   FFT: n2=1.54e-07 ninf=1.49e-07 < 8.11e-06 (0.018) 0 iFFT: n2=7.46e-02 ninf=2.16e-01 < 8.11e-06 (26665.475) 0 FAIL
pyopencl  R2C  (2,2,13000+2) axes=      [-1] ndim=None    float32 lut=True inplace=1  norm=   1 C   FFT: n2=1.54e-07 ninf=1.49e-07 < 8.11e-06 (0.018) 0 iFFT: n2=1.61e-07 ninf=4.19e-07 < 8.11e-06 (0.052) 0   OK
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   0 C   FFT: n2=1.54e-07 ninf=1.49e-07 < 4.06e-06 (0.037) 1 iFFT: n2=1.54e-07 ninf=3.75e-07 < 4.06e-06 (0.092) 1   OK
pyopencl  R2C    (2,2,13000) axes=      [-1] ndim=None    float32 lut=True inplace=0  norm=   1 C   FFT: n2=1.54e-07 ninf=1.49e-07 < 4.06e-06 (0.037) 1 iFFT: n2=1.61e-07 ninf=4.19e-07 < 4.06e-06 (0.103) 1   OK

.. and as you can see the results are slightly different.

So quite possibly not a VkFFT issue, but as you said compiler ?

@vincefn I think all the issues besides this 13000 R2C should be fixed. As for the 13000, does setting up forceCallbackVersionRealTransforms=1 help?

@vincefn I think all the issues besides this 13000 R2C should be fixed. As for the 13000, does setting up forceCallbackVersionRealTransforms=1 help?

Yes ! the quick r2c test (pyvkfft-test --r2c --nproc 8) passes (on the mac mini M1) with that option.

The new series on tests is ongoing:

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-a40cl/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-a40cu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-a100cu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-h100cl/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-h100cu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-apple-m1/pyvkfft-test.html (mac mini)

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-gtx1080ticu/pyvkfft-test.html

http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-v100cu/pyvkfft-test.html (power9)

Apart from the 13000 R2C failures on the M1 (which can be fixed with forceCallbackVersionRealTransforms=1, I can add a hook for M1 for that), it looks rather good. There is a small number of memory or timeout errors which can probably be ignored( I thought I had fixed the too many parallel process for DCTs).

Two new failures:

on the M1 for large DCT2/3 - non-radix: sizes 521, 541, 547 have a an accuracy failure after iFFT
on the A100 (cuda) the 1D DCT4 double-precision has 25 failures on the forward transform between lengths 2889 and 9835. This does not affect the A40 or the H100 (and I did not run the A100 with OpenCL). This was for an out-of-place transform, the inplace one will be done later.

@vincefn I believe b1ff5c4 fixed dct2/3 fails - pyvkfft passes the test on my machine with it and fails with the version before. I will add the 13000 "fix" to opencl and metal for Apple devices. I will look into the remaining 25 failures now.

I also couldn't verify the a100 failures (I used the 80GB A100 version, all 25 systems passed). Can you send the 2889 version output of pyvkfft with keepShaderCode = 1?

Here's the code - it's also an A100 80GB

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100 80G...  On   | 00000000:41:00.0 Off |                    0 |
| N/A   43C    P0    65W / 300W |    418MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+

I'm wondering about the 12.0 driver though - could be an early 12.x bug

dct4_float64_outofplace_norm1_a100.txt

dct4_2889_fp64_a100_12p2.txt This is the log from CUDA 12.2 - it matches, so must be a driver issue.

Actually, it looks like a toolkit issue - on the same A100 machine with driver 525.125.06 (cuda 12.0), there are the 26 DCT4 float64 failures using the 11.7.1 toolkit, but not using the 12.3.1 or 12.2.2 one.

@DTolm I just tested https://github.com/DTolm/VkFFT/commit/f8713188c5351b8b451ea25f29c86ea89e56139d on my macbook and it seems to have a lot of malloc issues - it segfaults almost immediately on the quick test pyvkfft-test --serial --r2c (the first sizes tested are with n=15).

Also the code workaround only refers to the 13000 size, but it's probably more complicated:

it is part of the quick test which only tests a few specific sizes
it only occurs on batch transforms - transforms on arrays of shape (13000) pass but there are errors on arrays of shape e.g. (2,2,13000) or (b,13000) but only if b>3

in fact with a shape of (b=10, n), testing without the, it fails for a number of sizes:


from pyvkfft.accuracy import test_accuracy
import numpy as np
for i in range(12900,13100,1):
print(test_accuracy('pyopencl',(10,i,),ndim=1,axes=None,dtype=np.float32,norm=0,inplace=1,r2c=True, use_lut=None)['str'])

pyopencl R2C (10,12900+2) axes= pyopencl R2C (10,12901+1) axes= pyopencl R2C (10,12902+2) axes= pyopencl R2C (10,12903+1) axes= pyopencl R2C (10,12904+2) axes= pyopencl R2C (10,12905+1) axes= pyopencl R2C (10,12906+2) axes= pyopencl R2C (10,12907+1) axes= pyopencl R2C (10,12908+2) axes= pyopencl R2C (10,12909+1) axes= pyopencl R2C (10,12910+2) axes= pyopencl R2C (10,12911+1) axes= pyopencl R2C (10,12912+2) axes= pyopencl R2C (10,12913+1) axes= pyopencl R2C (10,12914+2) axes= pyopencl R2C (10,12915+1) axes= pyopencl R2C (10,12916+2) axes= pyopencl R2C (10,12917+1) axes= pyopencl R2C (10,12918+2) axes= pyopencl R2C (10,12919+1) axes= pyopencl R2C (10,12920+2) axes= pyopencl R2C (10,12921+1) axes= pyopencl R2C (10,12922+2) axes= pyopencl R2C (10,12923+1) axes= pyopencl R2C (10,12924+2) axes= pyopencl R2C (10,12925+1) axes= pyopencl R2C (10,12926+2) axes= pyopencl R2C (10,12927+1) axes= pyopencl R2C (10,12928+2) axes= pyopencl R2C (10,12929+1) axes= pyopencl R2C (10,12930+2) axes= pyopencl R2C (10,12931+1) axes= pyopencl R2C (10,12932+2) axes= pyopencl R2C (10,12933+1) axes= pyopencl R2C (10,12934+2) axes= pyopencl R2C (10,12935+1) axes= pyopencl R2C (10,12936+2) axes= pyopencl R2C (10,12937+1) axes= pyopencl R2C (10,12938+2) axes= pyopencl R2C (10,12939+1) axes= pyopencl R2C (10,12940+2) axes= pyopencl R2C (10,12941+1) axes= pyopencl R2C (10,12942+2) axes= pyopencl R2C (10,12943+1) axes= pyopencl R2C (10,12944+2) axes= pyopencl R2C (10,12945+1) axes= pyopencl R2C (10,12946+2) axes= pyopencl R2C (10,12947+1) axes= pyopencl R2C (10,12948+2) axes= pyopencl R2C (10,12949+1) axes= pyopencl R2C (10,12950+2) axes= pyopencl R2C (10,12951+1) axes= pyopencl R2C (10,12952+2) axes= pyopencl R2C (10,12953+1) axes= pyopencl R2C (10,12954+2) axes= pyopencl R2C (10,12955+1) axes= pyopencl R2C (10,12956+2) axes= pyopencl R2C (10,12957+1) axes= pyopencl R2C (10,12958+2) axes= pyopencl R2C (10,12959+1) axes= pyopencl R2C (10,12960+2) axes= pyopencl R2C (10,12961+1) axes= pyopencl R2C (10,12962+2) axes= pyopencl R2C (10,12963+1) axes= pyopencl R2C (10,12964+2) axes= pyopencl R2C (10,12965+1) axes= pyopencl R2C (10,12966+2) axes= pyopencl R2C (10,12967+1) axes= pyopencl R2C (10,12968+2) axes= pyopencl R2C (10,12969+1) axes= pyopencl R2C (10,12970+2) axes= pyopencl R2C (10,12971+1) axes= pyopencl R2C (10,12972+2) axes= pyopencl R2C (10,12973+1) axes= pyopencl R2C (10,12974+2) axes= pyopencl R2C (10,12975+1) axes= pyopencl R2C (10,12976+2) axes= pyopencl R2C (10,12977+1) axes= pyopencl R2C (10,12978+2) axes= pyopencl R2C (10,12979+1) axes= pyopencl R2C (10,12980+2) axes= pyopencl R2C (10,12981+1) axes= pyopencl R2C (10,12982+2) axes= pyopencl R2C (10,12983+1) axes= pyopencl R2C (10,12984+2) axes= pyopencl R2C (10,12985+1) axes= pyopencl R2C (10,12986+2) axes= pyopencl R2C (10,12987+1) axes= pyopencl R2C (10,12988+2) axes= pyopencl R2C (10,12989+1) axes= pyopencl R2C (10,12990+2) axes= pyopencl R2C (10,12991+1) axes= pyopencl R2C (10,12992+2) axes= pyopencl R2C (10,12993+1) axes= pyopencl R2C (10,12994+2) axes= pyopencl R2C (10,12995+1) axes= pyopencl R2C (10,12996+2) axes= pyopencl R2C (10,12997+1) axes= pyopencl R2C (10,12998+2) axes= pyopencl R2C (10,12999+1) axes= pyopencl R2C (10,13000+2) axes= pyopencl R2C (10,13001+1) axes= pyopencl R2C (10,13002+2) axes= pyopencl R2C (10,13003+1) axes= pyopencl R2C (10,13004+2) axes= pyopencl R2C (10,13005+1) axes= pyopencl R2C (10,13006+2) axes= pyopencl R2C (10,13007+1) axes= pyopencl R2C (10,13008+2) axes= pyopencl R2C (10,13009+1) axes= pyopencl R2C (10,13010+2) axes= pyopencl R2C (10,13011+1) axes= pyopencl R2C (10,13012+2) axes= pyopencl R2C (10,13013+1) axes= pyopencl R2C (10,13014+2) axes= pyopencl R2C (10,13015+1) axes= pyopencl R2C (10,13016+2) axes= pyopencl R2C (10,13017+1) axes= pyopencl R2C (10,13018+2) axes= pyopencl R2C (10,13019+1) axes= pyopencl R2C (10,13020+2) axes= pyopencl R2C (10,13021+1) axes= pyopencl R2C (10,13022+2) axes= pyopencl R2C (10,13023+1) axes= pyopencl R2C (10,13024+2) axes= pyopencl R2C (10,13025+1) axes= pyopencl R2C (10,13026+2) axes= pyopencl R2C (10,13027+1) axes= pyopencl R2C (10,13028+2) axes= pyopencl R2C (10,13029+1) axes= pyopencl R2C (10,13030+2) axes= pyopencl R2C (10,13031+1) axes= pyopencl R2C (10,13032+2) axes= pyopencl R2C (10,13033+1) axes= pyopencl R2C (10,13034+2) axes= pyopencl R2C (10,13035+1) axes= pyopencl R2C (10,13036+2) axes= pyopencl R2C (10,13037+1) axes= pyopencl R2C (10,13038+2) axes= pyopencl R2C (10,13039+1) axes= pyopencl R2C (10,13040+2) axes= pyopencl R2C (10,13041+1) axes= pyopencl R2C (10,13042+2) axes= pyopencl R2C (10,13043+1) axes= pyopencl R2C (10,13044+2) axes= pyopencl R2C (10,13045+1) axes= pyopencl R2C (10,13046+2) axes= pyopencl R2C (10,13047+1) axes= pyopencl R2C (10,13048+2) axes= pyopencl R2C (10,13049+1) axes= pyopencl R2C (10,13050+2) axes= pyopencl R2C (10,13051+1) axes= pyopencl R2C (10,13052+2) axes= pyopencl R2C (10,13053+1) axes= pyopencl R2C (10,13054+2) axes= pyopencl R2C (10,13055+1) axes= pyopencl R2C (10,13056+2) axes= pyopencl R2C (10,13057+1) axes= pyopencl R2C (10,13058+2) axes= pyopencl R2C (10,13059+1) axes= pyopencl R2C (10,13060+2) axes= pyopencl R2C (10,13061+1) axes= pyopencl R2C (10,13062+2) axes= pyopencl R2C (10,13063+1) axes= pyopencl R2C (10,13064+2) axes= pyopencl R2C (10,13065+1) axes= pyopencl R2C (10,13066+2) axes= pyopencl R2C (10,13067+1) axes= pyopencl R2C (10,13068+2) axes= pyopencl R2C (10,13069+1) axes= pyopencl R2C (10,13070+2) axes= pyopencl R2C (10,13071+1) axes= pyopencl R2C (10,13072+2) axes= pyopencl R2C (10,13073+1) axes= pyopencl R2C (10,13074+2) axes= pyopencl R2C (10,13075+1) axes= pyopencl R2C (10,13076+2) axes= pyopencl R2C (10,13077+1) axes= pyopencl R2C (10,13078+2) axes= pyopencl R2C (10,13079+1) axes= pyopencl R2C (10,13080+2) axes= pyopencl R2C (10,13081+1) axes= pyopencl R2C (10,13082+2) axes= pyopencl R2C (10,13083+1) axes= pyopencl R2C (10,13084+2) axes= pyopencl R2C (10,13085+1) axes= pyopencl R2C (10,13086+2) axes= pyopencl R2C (10,13087+1) axes= pyopencl R2C (10,13088+2) axes= pyopencl R2C (10,13089+1) axes= pyopencl R2C (10,13090+2) axes= pyopencl R2C (10,13091+1) axes= pyopencl R2C (10,13092+2) axes= pyopencl R2C (10,13093+1) axes= pyopencl R2C (10,13094+2) axes= pyopencl R2C (10,13095+1) axes= pyopencl R2C (10,13096+2) axes= pyopencl R2C (10,13097+1) axes= pyopencl R2C (10,13098+2) axes= pyopencl R2C (10,13099+1) axes= None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.53e-07 ninf=2.79e-07 < 8.11e-06 (0.034) 0 iFFT: n2=2.47e-07 ninf=6.65e-07 < 8.11e-06 (0.082) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.80e-07 ninf=3.78e-07 < 8.11e-06 (0.047) 0 iFFT: n2=3.31e-07 ninf=7.73e-07 < 8.11e-06 (0.095) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.59e-07 ninf=6.51e-07 < 8.11e-06 (0.080) 0 iFFT: n2=5.58e-07 ninf=1.11e-06 < 8.11e-06 (0.137) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.32e-07 ninf=2.77e-07 < 8.11e-06 (0.034) 0 iFFT: n2=1.87e-07 ninf=5.68e-07 < 8.11e-06 (0.070) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.97e-07 ninf=7.33e-07 < 8.11e-06 (0.090) 0 iFFT: n2=5.97e-07 ninf=1.19e-06 < 8.11e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.78e-07 ninf=2.79e-07 < 8.11e-06 (0.034) 0 iFFT: n2=2.26e-07 ninf=6.06e-07 < 8.11e-06 (0.075) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.54e-07 ninf=6.12e-07 < 8.11e-06 (0.075) 0 iFFT: n2=5.55e-07 ninf=1.12e-06 < 8.11e-06 (0.138) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.81e-07 ninf=6.59e-07 < 8.11e-06 (0.081) 0 iFFT: n2=5.48e-07 ninf=1.01e-06 < 8.11e-06 (0.124) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.32e-07 ninf=5.27e-07 < 8.11e-06 (0.065) 0 iFFT: n2=5.32e-07 ninf=1.13e-06 < 8.11e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.59e-07 ninf=2.99e-07 < 8.11e-06 (0.037) 0 iFFT: n2=2.00e-07 ninf=5.34e-07 < 8.11e-06 (0.066) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.18e-07 ninf=6.91e-07 < 8.11e-06 (0.085) 0 iFFT: n2=6.16e-07 ninf=1.19e-06 < 8.11e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.62e-07 ninf=6.84e-07 < 8.11e-06 (0.084) 0 iFFT: n2=5.31e-07 ninf=9.60e-07 < 8.11e-06 (0.118) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.35e-07 ninf=5.63e-07 < 8.11e-06 (0.069) 0 iFFT: n2=5.35e-07 ninf=1.12e-06 < 8.11e-06 (0.138) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.78e-07 ninf=5.54e-07 < 8.11e-06 (0.068) 0 iFFT: n2=5.50e-07 ninf=9.93e-07 < 8.11e-06 (0.122) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.56e-07 ninf=5.56e-07 < 8.11e-06 (0.069) 0 iFFT: n2=5.55e-07 ninf=1.10e-06 < 8.11e-06 (0.136) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.60e-07 ninf=3.08e-07 < 8.11e-06 (0.038) 0 iFFT: n2=1.95e-07 ninf=5.36e-07 < 8.11e-06 (0.066) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.98e-07 ninf=6.30e-07 < 8.11e-06 (0.078) 0 iFFT: n2=5.97e-07 ninf=1.28e-06 < 8.11e-06 (0.158) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.66e-07 ninf=7.94e-07 < 8.11e-06 (0.098) 0 iFFT: n2=5.35e-07 ninf=9.53e-07 < 8.11e-06 (0.117) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.55e-07 ninf=5.97e-07 < 8.11e-06 (0.074) 0 iFFT: n2=5.57e-07 ninf=1.16e-06 < 8.11e-06 (0.143) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.31e-07 ninf=6.79e-07 < 8.11e-06 (0.084) 0 iFFT: n2=6.03e-07 ninf=1.05e-06 < 8.11e-06 (0.129) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.79e-07 ninf=3.53e-07 < 8.11e-06 (0.044) 0 iFFT: n2=7.00e-01 ninf=5.02e+00 < 8.11e-06 (618829.781) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.14e-07 ninf=7.48e-07 < 8.11e-06 (0.092) 0 iFFT: n2=5.86e-07 ninf=1.06e-06 < 8.11e-06 (0.130) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.49e-07 ninf=3.51e-07 < 8.11e-06 (0.043) 0 iFFT: n2=3.30e-01 ninf=2.23e+00 < 8.11e-06 (274548.040) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.16e-07 ninf=6.91e-07 < 8.11e-06 (0.085) 0 iFFT: n2=5.88e-07 ninf=1.02e-06 < 8.11e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.38e-07 ninf=6.83e-07 < 8.11e-06 (0.084) 0 iFFT: n2=5.38e-07 ninf=1.13e-06 < 8.11e-06 (0.139) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.33e-07 ninf=7.09e-07 < 8.11e-06 (0.087) 0 iFFT: n2=6.05e-07 ninf=1.06e-06 < 8.11e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.01e-07 ninf=3.67e-07 < 8.11e-06 (0.045) 0 iFFT: n2=2.99e-07 ninf=7.79e-07 < 8.11e-06 (0.096) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.75e-07 ninf=6.52e-07 < 8.11e-06 (0.080) 0 iFFT: n2=5.45e-07 ninf=1.03e-06 < 8.11e-06 (0.127) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.83e-07 ninf=4.08e-07 < 8.11e-06 (0.050) 0 iFFT: n2=2.85e-07 ninf=1.01e-06 < 8.11e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.25e-07 ninf=6.08e-07 < 8.11e-06 (0.075) 0 iFFT: n2=5.98e-07 ninf=1.10e-06 < 8.11e-06 (0.135) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.73e-07 ninf=5.98e-07 < 8.11e-06 (0.074) 0 iFFT: n2=5.73e-07 ninf=1.17e-06 < 8.11e-06 (0.145) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.19e-07 ninf=3.34e-07 < 8.11e-06 (0.041) 0 iFFT: n2=2.71e-07 ninf=6.62e-07 < 8.11e-06 (0.082) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.66e-07 ninf=3.03e-07 < 8.11e-06 (0.037) 0 iFFT: n2=2.67e-07 ninf=7.76e-07 < 8.11e-06 (0.096) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.84e-07 ninf=5.80e-07 < 8.11e-06 (0.072) 0 iFFT: n2=5.54e-07 ninf=1.01e-06 < 8.11e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.46e-07 ninf=5.92e-07 < 4.06e-06 (0.146) 0 iFFT: n2=5.46e-07 ninf=1.09e-06 < 4.06e-06 (0.268) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.78e-07 ninf=3.22e-07 < 4.06e-06 (0.079) 0 iFFT: n2=2.21e-07 ninf=5.83e-07 < 4.06e-06 (0.144) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.66e-07 ninf=5.32e-07 < 8.11e-06 (0.066) 0 iFFT: n2=2.18e-01 ninf=1.35e+00 < 8.11e-06 (166677.430) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.06e-07 ninf=6.77e-07 < 8.11e-06 (0.083) 0 iFFT: n2=5.79e-07 ninf=1.02e-06 < 8.11e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.85e-07 ninf=7.62e-07 < 8.11e-06 (0.094) 0 iFFT: n2=5.85e-07 ninf=1.13e-06 < 8.11e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.43e-07 ninf=8.13e-07 < 8.11e-06 (0.100) 0 iFFT: n2=6.15e-07 ninf=1.05e-06 < 8.11e-06 (0.129) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.34e-07 ninf=5.84e-07 < 8.11e-06 (0.072) 0 iFFT: n2=5.34e-07 ninf=1.14e-06 < 8.11e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.95e-07 ninf=6.02e-07 < 8.11e-06 (0.074) 0 iFFT: n2=5.68e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.95e-07 ninf=7.05e-07 < 8.11e-06 (0.087) 0 iFFT: n2=5.95e-07 ninf=1.11e-06 < 8.11e-06 (0.137) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.07e-07 ninf=3.49e-07 < 8.11e-06 (0.043) 0 iFFT: n2=2.51e-07 ninf=6.57e-07 < 8.11e-06 (0.081) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.81e-07 ninf=7.19e-07 < 8.11e-06 (0.089) 0 iFFT: n2=5.79e-07 ninf=1.08e-06 < 8.11e-06 (0.133) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.61e-07 ninf=5.33e-07 < 8.11e-06 (0.066) 0 iFFT: n2=5.30e-07 ninf=1.02e-06 < 8.11e-06 (0.126) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.70e-07 ninf=6.35e-07 < 8.11e-06 (0.078) 0 iFFT: n2=5.70e-07 ninf=1.23e-06 < 8.11e-06 (0.151) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.64e-07 ninf=6.16e-07 < 8.11e-06 (0.076) 0 iFFT: n2=5.33e-07 ninf=9.56e-07 < 8.11e-06 (0.118) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.83e-07 ninf=5.54e-07 < 8.11e-06 (0.068) 0 iFFT: n2=5.86e-07 ninf=1.14e-06 < 8.11e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.87e-07 ninf=5.82e-07 < 8.11e-06 (0.072) 0 iFFT: n2=5.57e-07 ninf=9.98e-07 < 8.11e-06 (0.123) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.06e-07 ninf=3.71e-07 < 8.11e-06 (0.046) 0 iFFT: n2=3.06e-07 ninf=8.02e-07 < 8.11e-06 (0.099) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.23e-07 ninf=6.31e-07 < 8.11e-06 (0.078) 0 iFFT: n2=5.93e-07 ninf=1.11e-06 < 8.11e-06 (0.136) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.64e-07 ninf=6.85e-07 < 8.11e-06 (0.084) 0 iFFT: n2=5.65e-07 ninf=1.14e-06 < 8.11e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.81e-07 ninf=6.65e-07 < 8.11e-06 (0.082) 0 iFFT: n2=5.51e-07 ninf=9.59e-07 < 8.11e-06 (0.118) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.01e-07 ninf=3.21e-07 < 8.11e-06 (0.040) 0 iFFT: n2=3.00e-07 ninf=8.65e-07 < 8.11e-06 (0.107) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.53e-07 ninf=7.03e-07 < 8.11e-06 (0.087) 0 iFFT: n2=6.24e-07 ninf=1.15e-06 < 8.11e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.03e-07 ninf=3.41e-07 < 8.11e-06 (0.042) 0 iFFT: n2=4.49e-01 ninf=2.69e+00 < 8.11e-06 (332038.360) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.43e-07 ninf=6.11e-07 < 8.11e-06 (0.075) 0 iFFT: n2=6.16e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.30e-07 ninf=4.78e-07 < 4.06e-06 (0.118) 0 iFFT: n2=3.25e-07 ninf=8.51e-07 < 4.06e-06 (0.210) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.46e-07 ninf=6.59e-07 < 4.06e-06 (0.163) 0 iFFT: n2=6.20e-07 ninf=1.08e-06 < 4.06e-06 (0.267) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=4.14e-07 ninf=5.05e-07 < 8.11e-06 (0.062) 0 iFFT: n2=3.91e-07 ninf=1.13e-06 < 8.11e-06 (0.139) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.76e-07 ninf=6.47e-07 < 8.11e-06 (0.080) 0 iFFT: n2=5.45e-07 ninf=1.00e-06 < 8.11e-06 (0.123) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.54e-07 ninf=5.88e-07 < 8.11e-06 (0.072) 0 iFFT: n2=5.52e-07 ninf=1.18e-06 < 8.11e-06 (0.145) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.13e-07 ninf=6.54e-07 < 8.11e-06 (0.081) 0 iFFT: n2=5.82e-07 ninf=1.04e-06 < 8.11e-06 (0.128) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.33e-07 ninf=3.84e-07 < 8.11e-06 (0.047) 0 iFFT: n2=3.34e-07 ninf=9.61e-07 < 8.11e-06 (0.118) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.69e-07 ninf=5.63e-07 < 8.11e-06 (0.069) 0 iFFT: n2=5.39e-07 ninf=9.68e-07 < 8.11e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.69e-07 ninf=6.17e-07 < 8.11e-06 (0.076) 0 iFFT: n2=5.69e-07 ninf=1.13e-06 < 8.11e-06 (0.139) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.45e-07 ninf=7.14e-07 < 8.11e-06 (0.088) 0 iFFT: n2=6.15e-07 ninf=1.07e-06 < 8.11e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.50e-07 ninf=5.58e-07 < 8.11e-06 (0.069) 0 iFFT: n2=5.51e-07 ninf=1.19e-06 < 8.11e-06 (0.147) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.95e-07 ninf=3.54e-07 < 8.11e-06 (0.044) 0 iFFT: n2=2.35e-07 ninf=6.12e-07 < 8.11e-06 (0.075) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.49e-07 ninf=5.61e-07 < 8.11e-06 (0.069) 0 iFFT: n2=5.49e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.70e-07 ninf=3.82e-07 < 8.11e-06 (0.047) 0 iFFT: n2=3.19e-07 ninf=7.18e-07 < 8.11e-06 (0.088) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.69e-07 ninf=5.76e-07 < 8.11e-06 (0.071) 0 iFFT: n2=5.66e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.69e-07 ninf=6.39e-07 < 8.11e-06 (0.079) 0 iFFT: n2=5.37e-07 ninf=9.66e-07 < 8.11e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.99e-07 ninf=6.88e-07 < 8.11e-06 (0.085) 0 iFFT: n2=5.97e-07 ninf=1.12e-06 < 8.11e-06 (0.138) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.08e-07 ninf=7.19e-07 < 8.11e-06 (0.089) 0 iFFT: n2=5.76e-07 ninf=9.80e-07 < 8.11e-06 (0.121) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.50e-07 ninf=6.61e-07 < 8.11e-06 (0.081) 0 iFFT: n2=5.50e-07 ninf=1.09e-06 < 8.11e-06 (0.134) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.67e-07 ninf=6.92e-07 < 8.11e-06 (0.085) 0 iFFT: n2=5.35e-07 ninf=9.64e-07 < 8.11e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.14e-07 ninf=6.31e-07 < 8.11e-06 (0.078) 0 iFFT: n2=6.15e-07 ninf=1.19e-06 < 8.11e-06 (0.147) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.39e-07 ninf=7.37e-07 < 8.11e-06 (0.091) 0 iFFT: n2=6.11e-07 ninf=1.08e-06 < 8.11e-06 (0.133) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.96e-07 ninf=6.68e-07 < 8.11e-06 (0.082) 0 iFFT: n2=5.94e-07 ninf=1.26e-06 < 8.11e-06 (0.156) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.29e-07 ninf=7.32e-07 < 8.11e-06 (0.090) 0 iFFT: n2=6.01e-07 ninf=1.09e-06 < 8.11e-06 (0.135) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.95e-07 ninf=6.95e-07 < 8.11e-06 (0.086) 0 iFFT: n2=5.93e-07 ninf=1.21e-06 < 8.11e-06 (0.149) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.38e-07 ninf=6.89e-07 < 8.11e-06 (0.085) 0 iFFT: n2=6.09e-07 ninf=1.06e-06 < 8.11e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.15e-07 ninf=7.65e-07 < 8.11e-06 (0.094) 0 iFFT: n2=6.14e-07 ninf=1.36e-06 < 8.11e-06 (0.168) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.57e-07 ninf=3.10e-07 < 8.11e-06 (0.038) 0 iFFT: n2=1.76e-07 ninf=5.41e-07 < 8.11e-06 (0.067) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.29e-07 ninf=3.59e-07 < 8.11e-06 (0.044) 0 iFFT: n2=2.55e-01 ninf=2.00e+00 < 8.11e-06 (246348.281) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.84e-07 ninf=3.14e-07 < 8.11e-06 (0.039) 0 iFFT: n2=2.36e-07 ninf=6.23e-07 < 8.11e-06 (0.077) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.98e-07 ninf=6.77e-07 < 8.11e-06 (0.083) 0 iFFT: n2=5.97e-07 ninf=1.14e-06 < 8.11e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.63e-07 ninf=6.33e-07 < 8.11e-06 (0.078) 0 iFFT: n2=5.35e-07 ninf=9.92e-07 < 8.11e-06 (0.122) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.07e-07 ninf=3.71e-07 < 8.11e-06 (0.046) 0 iFFT: n2=2.23e-01 ninf=9.56e-01 < 8.11e-06 (117773.324) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.39e-07 ninf=7.30e-07 < 8.11e-06 (0.090) 0 iFFT: n2=6.12e-07 ninf=1.08e-06 < 8.11e-06 (0.133) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.30e-07 ninf=2.78e-07 < 8.11e-06 (0.034) 0 iFFT: n2=2.28e-07 ninf=5.86e-07 < 8.11e-06 (0.072) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.99e-07 ninf=3.72e-07 < 8.11e-06 (0.046) 0 iFFT: n2=2.37e-07 ninf=6.08e-07 < 8.11e-06 (0.075) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.99e-07 ninf=2.95e-07 < 8.11e-06 (0.036) 0 iFFT: n2=5.63e-02 ninf=2.48e-01 < 8.11e-06 (30524.519) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.43e-07 ninf=4.02e-07 < 8.11e-06 (0.050) 0 iFFT: n2=2.90e-07 ninf=6.84e-07 < 8.11e-06 (0.084) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.13e-07 ninf=3.63e-07 < 8.11e-06 (0.045) 0 iFFT: n2=7.38e-02 ninf=3.00e-01 < 8.11e-06 (36956.972) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.70e-07 ninf=6.93e-07 < 8.11e-06 (0.085) 0 iFFT: n2=5.38e-07 ninf=9.79e-07 < 8.11e-06 (0.121) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.61e-07 ninf=4.01e-07 < 4.06e-06 (0.099) 0 iFFT: n2=3.64e-07 ninf=8.96e-07 < 4.06e-06 (0.221) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.08e-07 ninf=6.70e-07 < 4.06e-06 (0.165) 0 iFFT: n2=5.78e-07 ninf=1.06e-06 < 4.06e-06 (0.261) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.37e-07 ninf=4.31e-07 < 8.11e-06 (0.053) 0 iFFT: n2=9.11e-02 ninf=4.95e-01 < 8.11e-06 (60996.984) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.69e-07 ninf=6.09e-07 < 8.11e-06 (0.075) 0 iFFT: n2=5.36e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.14e-07 ninf=3.89e-07 < 8.11e-06 (0.048) 0 iFFT: n2=2.77e-01 ninf=2.06e+00 < 8.11e-06 (253917.409) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.37e-07 ninf=6.35e-07 < 8.11e-06 (0.078) 0 iFFT: n2=6.13e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.98e-07 ninf=6.40e-07 < 8.11e-06 (0.079) 0 iFFT: n2=5.99e-07 ninf=1.19e-06 < 8.11e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.03e-07 ninf=3.65e-07 < 8.11e-06 (0.045) 0 iFFT: n2=2.69e-07 ninf=6.65e-07 < 8.11e-06 (0.082) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.00e-07 ninf=6.74e-07 < 8.11e-06 (0.083) 0 iFFT: n2=5.99e-07 ninf=1.38e-06 < 8.11e-06 (0.170) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.41e-07 ninf=7.21e-07 < 8.11e-06 (0.089) 0 iFFT: n2=6.14e-07 ninf=1.08e-06 < 8.11e-06 (0.134) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.50e-07 ninf=3.53e-07 < 8.11e-06 (0.043) 0 iFFT: n2=3.52e-07 ninf=9.49e-07 < 8.11e-06 (0.117) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.72e-07 ninf=6.45e-07 < 8.11e-06 (0.079) 0 iFFT: n2=5.42e-07 ninf=9.63e-07 < 8.11e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.54e-07 ninf=5.78e-07 < 8.11e-06 (0.071) 0 iFFT: n2=5.52e-07 ninf=1.15e-06 < 8.11e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.11e-07 ninf=6.11e-07 < 8.11e-06 (0.075) 0 iFFT: n2=5.82e-07 ninf=1.09e-06 < 8.11e-06 (0.134) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.02e-07 ninf=6.86e-07 < 8.11e-06 (0.085) 0 iFFT: n2=6.04e-07 ninf=1.13e-06 < 8.11e-06 (0.139) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.67e-07 ninf=5.79e-07 < 8.11e-06 (0.071) 0 iFFT: n2=2.48e-07 ninf=6.71e-07 < 8.11e-06 (0.083) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.88e-07 ninf=3.04e-07 < 8.11e-06 (0.037) 0 iFFT: n2=2.87e-07 ninf=7.31e-07 < 8.11e-06 (0.090) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.47e-07 ninf=7.56e-07 < 8.11e-06 (0.093) 0 iFFT: n2=6.19e-07 ninf=1.14e-06 < 8.11e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.60e-07 ninf=5.52e-07 < 8.11e-06 (0.068) 0 iFFT: n2=5.57e-07 ninf=1.11e-06 < 8.11e-06 (0.137) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.37e-07 ninf=7.09e-07 < 8.11e-06 (0.087) 0 iFFT: n2=6.11e-07 ninf=1.10e-06 < 8.11e-06 (0.136) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.55e-07 ninf=6.01e-07 < 8.11e-06 (0.074) 0 iFFT: n2=5.54e-07 ninf=1.18e-06 < 8.11e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.48e-07 ninf=7.13e-07 < 8.11e-06 (0.088) 0 iFFT: n2=6.20e-07 ninf=1.09e-06 < 8.11e-06 (0.134) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.65e-07 ninf=2.95e-07 < 8.11e-06 (0.036) 0 iFFT: n2=2.57e-07 ninf=6.99e-07 < 8.11e-06 (0.086) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.01e-07 ninf=3.46e-07 < 8.11e-06 (0.043) 0 iFFT: n2=2.49e-07 ninf=7.21e-07 < 8.11e-06 (0.089) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.01e-07 ninf=5.86e-07 < 8.11e-06 (0.072) 0 iFFT: n2=6.01e-07 ninf=1.31e-06 < 8.11e-06 (0.162) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.17e-07 ninf=6.26e-07 < 8.11e-06 (0.077) 0 iFFT: n2=5.87e-07 ninf=1.11e-06 < 8.11e-06 (0.137) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.84e-07 ninf=3.18e-07 < 8.11e-06 (0.039) 0 iFFT: n2=2.80e-07 ninf=7.86e-07 < 8.11e-06 (0.097) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.77e-07 ninf=6.73e-07 < 8.11e-06 (0.083) 0 iFFT: n2=5.43e-07 ninf=9.98e-07 < 8.11e-06 (0.123) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.29e-07 ninf=5.41e-07 < 8.11e-06 (0.067) 0 iFFT: n2=5.29e-07 ninf=1.07e-06 < 8.11e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.49e-07 ninf=7.35e-07 < 8.11e-06 (0.091) 0 iFFT: n2=6.21e-07 ninf=1.14e-06 < 8.11e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.04e-07 ninf=8.01e-07 < 8.11e-06 (0.099) 0 iFFT: n2=6.04e-07 ninf=1.18e-06 < 8.11e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.24e-07 ninf=3.60e-07 < 8.11e-06 (0.044) 0 iFFT: n2=2.59e-07 ninf=6.51e-07 < 8.11e-06 (0.080) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.09e-07 ninf=7.00e-07 < 8.11e-06 (0.086) 0 iFFT: n2=6.08e-07 ninf=1.31e-06 < 8.11e-06 (0.162) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.49e-07 ninf=7.39e-07 < 8.11e-06 (0.091) 0 iFFT: n2=6.24e-07 ninf=1.10e-06 < 8.11e-06 (0.135) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.65e-07 ninf=3.22e-07 < 8.12e-06 (0.040) 0 iFFT: n2=1.45e+00 ninf=1.02e+01 < 8.12e-06 (1260703.836) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.79e-07 ninf=6.89e-07 < 8.12e-06 (0.085) 0 iFFT: n2=5.49e-07 ninf=9.98e-07 < 8.12e-06 (0.123) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.50e-07 ninf=3.72e-07 < 8.12e-06 (0.046) 0 iFFT: n2=3.63e-01 ninf=2.18e+00 < 8.12e-06 (268743.397) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.67e-07 ninf=3.01e-07 < 8.12e-06 (0.037) 0 iFFT: n2=2.12e-07 ninf=5.56e-07 < 8.12e-06 (0.069) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.12e-07 ninf=7.01e-07 < 8.12e-06 (0.086) 0 iFFT: n2=6.11e-07 ninf=1.30e-06 < 8.12e-06 (0.161) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.82e-07 ninf=6.53e-07 < 8.12e-06 (0.080) 0 iFFT: n2=5.49e-07 ninf=9.56e-07 < 8.12e-06 (0.118) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.70e-07 ninf=3.18e-07 < 8.12e-06 (0.039) 0 iFFT: n2=2.69e-07 ninf=7.06e-07 < 8.12e-06 (0.087) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.52e-07 ninf=6.95e-07 < 8.12e-06 (0.086) 0 iFFT: n2=6.27e-07 ninf=1.15e-06 < 8.12e-06 (0.142) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.16e-07 ninf=3.33e-07 < 8.12e-06 (0.041) 0 iFFT: n2=3.16e-07 ninf=9.85e-07 < 8.12e-06 (0.121) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.67e-07 ninf=2.50e-07 < 8.12e-06 (0.031) 0 iFFT: n2=2.26e-07 ninf=6.34e-07 < 8.12e-06 (0.078) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.66e-07 ninf=5.31e-07 < 8.12e-06 (0.065) 0 iFFT: n2=5.63e-07 ninf=1.18e-06 < 8.12e-06 (0.145) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.52e-07 ninf=7.06e-07 < 8.12e-06 (0.087) 0 iFFT: n2=6.23e-07 ninf=1.07e-06 < 8.12e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.80e-07 ninf=6.43e-07 < 8.12e-06 (0.079) 0 iFFT: n2=5.78e-07 ninf=1.14e-06 < 8.12e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.86e-07 ninf=6.46e-07 < 8.12e-06 (0.080) 0 iFFT: n2=5.56e-07 ninf=1.08e-06 < 8.12e-06 (0.133) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.10e-07 ninf=6.44e-07 < 8.12e-06 (0.079) 0 iFFT: n2=6.09e-07 ninf=1.14e-06 < 8.12e-06 (0.140) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.21e-07 ninf=8.13e-07 < 8.12e-06 (0.100) 0 iFFT: n2=5.94e-07 ninf=1.07e-06 < 8.12e-06 (0.132) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.65e-07 ninf=5.55e-07 < 8.12e-06 (0.068) 0 iFFT: n2=5.64e-07 ninf=1.30e-06 < 8.12e-06 (0.161) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.80e-07 ninf=6.22e-07 < 8.12e-06 (0.077) 0 iFFT: n2=5.52e-07 ninf=9.98e-07 < 8.12e-06 (0.123) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.47e-07 ninf=2.71e-07 < 8.12e-06 (0.033) 0 iFFT: n2=2.42e-07 ninf=7.52e-07 < 8.12e-06 (0.093) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.71e-07 ninf=4.35e-07 < 8.12e-06 (0.054) 0 iFFT: n2=2.94e-07 ninf=7.09e-07 < 8.12e-06 (0.087) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.14e-07 ninf=3.77e-07 < 8.12e-06 (0.046) 0 iFFT: n2=3.13e-07 ninf=8.80e-07 < 8.12e-06 (0.108) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.49e-07 ninf=6.90e-07 < 8.12e-06 (0.085) 0 iFFT: n2=6.18e-07 ninf=1.06e-06 < 8.12e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.09e-07 ninf=7.01e-07 < 8.12e-06 (0.086) 0 iFFT: n2=6.12e-07 ninf=1.20e-06 < 8.12e-06 (0.148) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.51e-07 ninf=6.71e-07 < 8.12e-06 (0.083) 0 iFFT: n2=6.23e-07 ninf=1.06e-06 < 8.12e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.55e-07 ninf=4.00e-07 < 8.12e-06 (0.049) 0 iFFT: n2=2.56e-07 ninf=7.48e-07 < 8.12e-06 (0.092) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.81e-07 ninf=6.30e-07 < 8.12e-06 (0.078) 0 iFFT: n2=5.51e-07 ninf=9.68e-07 < 8.12e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.59e-07 ninf=6.40e-07 < 8.12e-06 (0.079) 0 iFFT: n2=5.60e-07 ninf=1.20e-06 < 8.12e-06 (0.147) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.20e-07 ninf=7.42e-07 < 8.12e-06 (0.091) 0 iFFT: n2=5.90e-07 ninf=1.04e-06 < 8.12e-06 (0.128) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.06e-07 ninf=7.04e-07 < 8.12e-06 (0.087) 0 iFFT: n2=6.07e-07 ninf=1.25e-06 < 8.12e-06 (0.154) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.33e-07 ninf=3.92e-07 < 8.12e-06 (0.048) 0 iFFT: n2=2.71e-07 ninf=6.56e-07 < 8.12e-06 (0.081) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.77e-07 ninf=6.83e-07 < 8.12e-06 (0.084) 0 iFFT: n2=5.78e-07 ninf=1.18e-06 < 8.12e-06 (0.145) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.50e-07 ninf=6.79e-07 < 8.12e-06 (0.084) 0 iFFT: n2=6.22e-07 ninf=1.11e-06 < 8.12e-06 (0.136) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.67e-07 ninf=3.35e-07 < 8.12e-06 (0.041) 0 iFFT: n2=3.60e-01 ninf=2.69e+00 < 8.12e-06 (331265.471) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.64e-07 ninf=3.17e-07 < 8.12e-06 (0.039) 0 iFFT: n2=2.22e-07 ninf=5.96e-07 < 8.12e-06 (0.073) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.57e-07 ninf=5.87e-07 < 4.06e-06 (0.145) 0 iFFT: n2=5.57e-07 ninf=1.31e-06 < 4.06e-06 (0.323) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.50e-07 ninf=6.79e-07 < 4.06e-06 (0.167) 0 iFFT: n2=6.21e-07 ninf=1.20e-06 < 4.06e-06 (0.295) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=4.10e-07 ninf=5.24e-07 < 8.12e-06 (0.065) 0 iFFT: n2=5.28e-01 ninf=3.68e+00 < 8.12e-06 (453799.101) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.74e-07 ninf=5.96e-07 < 8.12e-06 (0.073) 0 iFFT: n2=5.42e-07 ninf=1.02e-06 < 8.12e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.04e-07 ninf=6.65e-07 < 8.12e-06 (0.082) 0 iFFT: n2=6.01e-07 ninf=1.32e-06 < 8.12e-06 (0.162) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.15e-07 ninf=7.01e-07 < 8.12e-06 (0.086) 0 iFFT: n2=5.84e-07 ninf=1.02e-06 < 8.12e-06 (0.126) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.10e-07 ninf=3.36e-07 < 8.12e-06 (0.041) 0 iFFT: n2=2.63e-01 ninf=1.54e+00 < 8.12e-06 (189714.770) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.73e-07 ninf=5.98e-07 < 8.12e-06 (0.074) 0 iFFT: n2=5.41e-07 ninf=9.63e-07 < 8.12e-06 (0.119) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.14e-07 ninf=6.82e-07 < 8.12e-06 (0.084) 0 iFFT: n2=6.15e-07 ninf=1.20e-06 < 8.12e-06 (0.148) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.41e-07 ninf=7.03e-07 < 8.12e-06 (0.087) 0 iFFT: n2=6.13e-07 ninf=1.09e-06 < 8.12e-06 (0.134) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.97e-07 ninf=7.11e-07 < 8.12e-06 (0.088) 0 iFFT: n2=5.95e-07 ninf=1.25e-06 < 8.12e-06 (0.154) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.36e-07 ninf=7.18e-07 < 8.12e-06 (0.088) 0 iFFT: n2=6.05e-07 ninf=1.00e-06 < 8.12e-06 (0.124) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.98e-07 ninf=6.68e-07 < 8.12e-06 (0.082) 0 iFFT: n2=5.98e-07 ninf=1.19e-06 < 8.12e-06 (0.147) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.75e-07 ninf=3.48e-07 < 8.12e-06 (0.043) 0 iFFT: n2=2.19e-07 ninf=5.67e-07 < 8.12e-06 (0.070) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=3.27e-07 ninf=3.82e-07 < 8.12e-06 (0.047) 0 iFFT: n2=3.27e-07 ninf=7.72e-07 < 8.12e-06 (0.095) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.63e-07 ninf=6.07e-07 < 8.12e-06 (0.075) 0 iFFT: n2=5.32e-07 ninf=9.49e-07 < 8.12e-06 (0.117) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.98e-07 ninf=3.79e-07 < 8.12e-06 (0.047) 0 iFFT: n2=2.92e-07 ninf=7.52e-07 < 8.12e-06 (0.093) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.69e-07 ninf=3.12e-07 < 8.12e-06 (0.038) 0 iFFT: n2=2.11e-07 ninf=5.51e-07 < 8.12e-06 (0.068) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.87e-07 ninf=6.32e-07 < 8.12e-06 (0.078) 0 iFFT: n2=5.86e-07 ninf=1.12e-06 < 8.12e-06 (0.138) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.54e-07 ninf=6.50e-07 < 8.12e-06 (0.080) 0 iFFT: n2=6.23e-07 ninf=1.14e-06 < 8.12e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.56e-07 ninf=5.86e-07 < 8.12e-06 (0.072) 0 iFFT: n2=5.55e-07 ninf=1.14e-06 < 8.12e-06 (0.141) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.28e-07 ninf=6.65e-07 < 8.12e-06 (0.082) 0 iFFT: n2=5.97e-07 ninf=1.15e-06 < 8.12e-06 (0.142) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.35e-07 ninf=5.33e-07 < 8.12e-06 (0.066) 0 iFFT: n2=5.35e-07 ninf=1.13e-06 < 8.12e-06 (0.139) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.15e-07 ninf=6.41e-07 < 8.12e-06 (0.079) 0 iFFT: n2=5.88e-07 ninf=1.06e-06 < 8.12e-06 (0.131) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.25e-07 ninf=2.99e-07 < 8.12e-06 (0.037) 0 iFFT: n2=1.89e-01 ninf=1.03e+00 < 8.12e-06 (127352.008) 0 FAIL None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.75e-07 ninf=2.83e-07 < 8.12e-06 (0.035) 0 iFFT: n2=2.21e-07 ninf=5.46e-07 < 8.12e-06 (0.067) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.44e-07 ninf=5.60e-07 < 8.12e-06 (0.069) 0 iFFT: n2=5.43e-07 ninf=1.06e-06 < 8.12e-06 (0.130) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.41e-07 ninf=7.16e-07 < 8.12e-06 (0.088) 0 iFFT: n2=6.13e-07 ninf=1.08e-06 < 8.12e-06 (0.133) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.73e-07 ninf=6.06e-07 < 8.12e-06 (0.075) 0 iFFT: n2=5.73e-07 ninf=1.18e-06 < 8.12e-06 (0.145) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=2.81e-07 ninf=3.23e-07 < 8.12e-06 (0.040) 0 iFFT: n2=2.39e-07 ninf=6.57e-07 < 8.12e-06 (0.081) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.17e-07 ninf=7.23e-07 < 8.12e-06 (0.089) 0 iFFT: n2=6.16e-07 ninf=1.19e-06 < 8.12e-06 (0.147) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.34e-07 ninf=7.50e-07 < 8.12e-06 (0.092) 0 iFFT: n2=6.05e-07 ninf=1.01e-06 < 8.12e-06 (0.125) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=5.79e-07 ninf=6.81e-07 < 8.12e-06 (0.084) 0 iFFT: n2=5.82e-07 ninf=1.19e-06 < 8.12e-06 (0.146) 0 OK None ndim= 1 float32 lut=None inplace=1 norm= 0 C FFT: n2=6.05e-07 ninf=6.63e-07 < 8.12e-06 (0.082) 0 iFFT: n2=5.76e-07 ninf=1.06e-06 < 8.12e-06 (0.131) 0 OK


I tried all even sizes with a shape of (10,n) from n=8000 and the first failure was for 8338

and more interestingly, when tested individually (manually, not in a consecutive loop), the tests which fail in the loop will only fail like half of the time, randomly. Which indicates that the VkFFT calculations are OK but there may be some exotic compiler or driver issue.

Maybe it'd be better to avoid the bug by using `forceCallbackVersionRealTransforms=1` from the pyvkfft side, and avoid this kind of hook inside VkFFT - up to you.

@vincefn I have forced the forceCallbackVersionRealTransforms=1 for all big even sequences of R2C/C2R on Apple devices.

Thanks ! This fixes the "13000" tests and the segfault, but now r2c transforms with size 2988 (1D and 2D, in or out-of-place) are failing for opencl on the mac mini M1 (It passes using cuda and opencl on the H100 and A100):

In practice it starts failing at n=2060 when testing systematically (only non-radix sizes):

pyopencl  R2C        (2050,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.15e-07 ninf=2.39e-07 < 7.31e-06 (0.033) 1 iFFT: n2=1.93e-07 ninf=4.17e-07 < 7.31e-06 (0.057) 1   OK
pyopencl  R2C        (2051,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.56e-07 ninf=7.13e-07 < 7.31e-06 (0.098) 1 iFFT: n2=6.07e-07 ninf=1.01e-06 < 7.31e-06 (0.139) 1   OK
pyopencl  R2C        (2052,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.07e-07 ninf=2.31e-07 < 7.31e-06 (0.032) 1 iFFT: n2=1.84e-07 ninf=3.58e-07 < 7.31e-06 (0.049) 1   OK
pyopencl  R2C        (2053,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.33e-07 ninf=6.98e-07 < 7.31e-06 (0.095) 1 iFFT: n2=5.95e-07 ninf=1.07e-06 < 7.31e-06 (0.147) 1   OK
pyopencl  R2C        (2054,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.53e-07 ninf=4.08e-07 < 7.31e-06 (0.056) 1 iFFT: n2=1.92e-07 ninf=4.18e-07 < 7.31e-06 (0.057) 1   OK
pyopencl  R2C        (2055,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.68e-07 ninf=7.10e-07 < 7.31e-06 (0.097) 1 iFFT: n2=5.76e-07 ninf=1.01e-06 < 7.31e-06 (0.139) 1   OK
pyopencl  R2C        (2056,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.61e-07 ninf=3.36e-07 < 7.31e-06 (0.046) 1 iFFT: n2=2.39e-07 ninf=5.36e-07 < 7.31e-06 (0.073) 1   OK
pyopencl  R2C        (2057,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=1.93e-07 ninf=1.82e-07 < 7.31e-06 (0.025) 1 iFFT: n2=1.74e-07 ninf=3.58e-07 < 7.31e-06 (0.049) 1   OK
pyopencl  R2C        (2058,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.30e-07 ninf=3.71e-07 < 3.66e-06 (0.102) 1 iFFT: n2=2.70e-07 ninf=5.96e-07 < 3.66e-06 (0.163) 1   OK
pyopencl  R2C        (2059,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.53e-07 ninf=2.41e-07 < 7.31e-06 (0.033) 1 iFFT: n2=2.10e-07 ninf=4.17e-07 < 7.31e-06 (0.057) 1   OK
pyopencl  R2C        (2060,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=7.35e-07 ninf=8.71e-07 < 7.31e-06 (0.119) 1 iFFT: n2=1.03e+00 ninf=1.46e+00 < 7.31e-06 (199607.795) 1 FAIL
pyopencl  R2C        (2061,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.45e-07 ninf=6.24e-07 < 7.31e-06 (0.085) 1 iFFT: n2=6.23e-07 ninf=1.13e-06 < 7.31e-06 (0.155) 1   OK
pyopencl  R2C        (2062,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=7.24e-07 ninf=7.03e-07 < 7.31e-06 (0.096) 1 iFFT: n2=1.02e+00 ninf=1.62e+00 < 7.31e-06 (221178.044) 1 FAIL
pyopencl  R2C        (2063,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.33e-07 ninf=6.77e-07 < 7.31e-06 (0.093) 1 iFFT: n2=5.99e-07 ninf=1.01e-06 < 7.31e-06 (0.139) 1   OK
pyopencl  R2C        (2064,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.20e-07 ninf=2.06e-07 < 7.31e-06 (0.028) 1 iFFT: n2=2.06e-07 ninf=4.77e-07 < 7.31e-06 (0.065) 1   OK
pyopencl  R2C        (2065,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.63e-07 ninf=5.54e-07 < 7.31e-06 (0.076) 1 iFFT: n2=6.24e-07 ninf=1.13e-06 < 7.31e-06 (0.155) 1   OK
pyopencl  R2C        (2066,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.30e-07 ninf=5.85e-07 < 7.32e-06 (0.080) 1 iFFT: n2=1.04e+00 ninf=1.49e+00 < 7.32e-06 (204058.680) 1 FAIL
pyopencl  R2C        (2067,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.12e-07 ninf=2.20e-07 < 7.32e-06 (0.030) 1 iFFT: n2=1.80e-07 ninf=4.18e-07 < 7.32e-06 (0.057) 1   OK
pyopencl  R2C        (2068,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.43e-07 ninf=6.39e-07 < 7.32e-06 (0.087) 1 iFFT: n2=1.02e+00 ninf=1.66e+00 < 7.32e-06 (226299.890) 1 FAIL
pyopencl  R2C        (2069,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.33e-07 ninf=5.02e-07 < 7.32e-06 (0.069) 1 iFFT: n2=5.81e-07 ninf=1.13e-06 < 7.32e-06 (0.155) 1   OK
pyopencl  R2C        (2070,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.09e-07 ninf=2.41e-07 < 7.32e-06 (0.033) 1 iFFT: n2=1.83e-07 ninf=3.58e-07 < 7.32e-06 (0.049) 1   OK
pyopencl  R2C        (2071,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.91e-07 ninf=3.18e-07 < 7.32e-06 (0.043) 1 iFFT: n2=2.52e-07 ninf=5.96e-07 < 7.32e-06 (0.082) 1   OK
pyopencl  R2C        (2072,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.17e-07 ninf=2.55e-07 < 7.32e-06 (0.035) 1 iFFT: n2=1.95e-07 ninf=3.58e-07 < 7.32e-06 (0.049) 1   OK
pyopencl  R2C        (2073,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.31e-07 ninf=7.08e-07 < 7.32e-06 (0.097) 1 iFFT: n2=5.67e-07 ninf=1.01e-06 < 7.32e-06 (0.139) 1   OK
pyopencl  R2C        (2074,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.30e-07 ninf=2.82e-07 < 7.32e-06 (0.039) 1 iFFT: n2=2.08e-07 ninf=4.17e-07 < 7.32e-06 (0.057) 1   OK
pyopencl  R2C        (2075,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.05e-07 ninf=6.16e-07 < 7.32e-06 (0.084) 1 iFFT: n2=6.10e-07 ninf=1.01e-06 < 7.32e-06 (0.139) 1   OK
pyopencl  R2C        (2076,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=5.71e-07 ninf=6.92e-07 < 7.32e-06 (0.095) 1 iFFT: n2=1.03e+00 ninf=1.45e+00 < 7.32e-06 (198046.534) 1 FAIL
pyopencl  R2C        (2077,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.57e-07 ninf=2.76e-07 < 7.32e-06 (0.038) 1 iFFT: n2=2.25e-07 ninf=4.77e-07 < 7.32e-06 (0.065) 1   OK
pyopencl  R2C        (2078,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.38e-07 ninf=5.97e-07 < 7.32e-06 (0.082) 1 iFFT: n2=1.02e+00 ninf=1.76e+00 < 7.32e-06 (240842.954) 1 FAIL
pyopencl  R2C        (2079,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.49e-07 ninf=3.86e-07 < 3.66e-06 (0.106) 1 iFFT: n2=2.70e-07 ninf=5.37e-07 < 3.66e-06 (0.147) 1   OK
pyopencl  R2C        (2080,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=4.26e-07 < 3.66e-06 (0.116) 1 iFFT: n2=2.84e-07 ninf=5.96e-07 < 3.66e-06 (0.163) 1   OK
pyopencl  R2C        (2081,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.36e-07 ninf=3.15e-07 < 7.32e-06 (0.043) 1 iFFT: n2=2.54e-07 ninf=5.37e-07 < 7.32e-06 (0.073) 1   OK
pyopencl  R2C        (2082,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=7.03e-07 ninf=7.56e-07 < 7.32e-06 (0.103) 1 iFFT: n2=1.03e+00 ninf=1.49e+00 < 7.32e-06 (204262.549) 1 FAIL
pyopencl  R2C        (2083,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.83e-07 ninf=7.09e-07 < 7.32e-06 (0.097) 1 iFFT: n2=6.28e-07 ninf=1.19e-06 < 7.32e-06 (0.163) 1   OK
pyopencl  R2C        (2084,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.22e-07 ninf=3.06e-07 < 7.32e-06 (0.042) 1 iFFT: n2=2.41e-07 ninf=4.77e-07 < 7.32e-06 (0.065) 1   OK
pyopencl  R2C        (2085,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.59e-07 ninf=5.96e-07 < 7.32e-06 (0.081) 1 iFFT: n2=6.29e-07 ninf=1.01e-06 < 7.32e-06 (0.138) 1   OK
pyopencl  R2C        (2086,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.47e-07 ninf=7.27e-07 < 7.32e-06 (0.099) 1 iFFT: n2=1.03e+00 ninf=1.41e+00 < 7.32e-06 (192035.003) 1 FAIL
pyopencl  R2C        (2087,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.93e-07 ninf=7.80e-07 < 7.32e-06 (0.107) 1 iFFT: n2=5.82e-07 ninf=1.07e-06 < 7.32e-06 (0.147) 1   OK
pyopencl  R2C        (2088,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.11e-07 ninf=2.24e-07 < 7.32e-06 (0.031) 1 iFFT: n2=1.87e-07 ninf=3.58e-07 < 7.32e-06 (0.049) 1   OK
pyopencl  R2C        (2089,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=7.63e-07 ninf=8.29e-07 < 7.32e-06 (0.113) 1 iFFT: n2=5.93e-07 ninf=9.54e-07 < 7.32e-06 (0.130) 1   OK
pyopencl  R2C        (2090,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.15e-07 ninf=2.40e-07 < 7.32e-06 (0.033) 1 iFFT: n2=2.02e-07 ninf=4.17e-07 < 7.32e-06 (0.057) 1   OK
pyopencl  R2C        (2091,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.31e-07 ninf=2.26e-07 < 7.32e-06 (0.031) 1 iFFT: n2=1.95e-07 ninf=3.87e-07 < 7.32e-06 (0.053) 1   OK
pyopencl  R2C        (2092,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.94e-07 ninf=9.59e-07 < 7.32e-06 (0.131) 1 iFFT: n2=1.01e+00 ninf=1.39e+00 < 7.32e-06 (190553.593) 1 FAIL
pyopencl  R2C        (2093,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.11e-07 ninf=2.16e-07 < 7.32e-06 (0.030) 1 iFFT: n2=1.75e-07 ninf=4.77e-07 < 7.32e-06 (0.065) 1   OK
pyopencl  R2C        (2094,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.76e-07 ninf=6.56e-07 < 7.32e-06 (0.090) 1 iFFT: n2=1.03e+00 ninf=1.54e+00 < 7.32e-06 (210413.452) 1 FAIL
pyopencl  R2C        (2095,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.29e-07 ninf=5.38e-07 < 7.32e-06 (0.074) 1 iFFT: n2=6.68e-07 ninf=1.13e-06 < 7.32e-06 (0.155) 1   OK
pyopencl  R2C        (2096,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.38e-07 ninf=2.88e-07 < 7.32e-06 (0.039) 1 iFFT: n2=2.00e-07 ninf=4.17e-07 < 7.32e-06 (0.057) 1   OK
pyopencl  R2C        (2097,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.48e-07 ninf=7.78e-07 < 7.32e-06 (0.106) 1 iFFT: n2=6.05e-07 ninf=1.19e-06 < 7.32e-06 (0.163) 1   OK
pyopencl  R2C        (2098,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.95e-07 ninf=7.81e-07 < 7.32e-06 (0.107) 1 iFFT: n2=1.02e+00 ninf=1.49e+00 < 7.32e-06 (203305.024) 1 FAIL
pyopencl  R2C        (2099,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.96e-07 ninf=6.43e-07 < 7.32e-06 (0.088) 1 iFFT: n2=6.17e-07 ninf=1.13e-06 < 7.32e-06 (0.155) 1   OK
pyopencl  R2C        (2100,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.41e-07 ninf=3.81e-07 < 3.66e-06 (0.104) 1 iFFT: n2=2.79e-07 ninf=5.96e-07 < 3.66e-06 (0.163) 1   OK
pyopencl  R2C        (2101,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.60e-07 ninf=6.77e-07 < 7.32e-06 (0.092) 1 iFFT: n2=6.33e-07 ninf=1.13e-06 < 7.32e-06 (0.155) 1   OK
pyopencl  R2C        (2102,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.25e-07 ninf=3.74e-07 < 7.32e-06 (0.051) 1 iFFT: n2=2.53e-07 ninf=5.37e-07 < 7.32e-06 (0.073) 1   OK
pyopencl  R2C        (2103,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.37e-07 ninf=3.06e-07 < 7.32e-06 (0.042) 1 iFFT: n2=2.44e-07 ninf=5.36e-07 < 7.32e-06 (0.073) 1   OK
pyopencl  R2C        (2104,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=6.64e-07 ninf=7.71e-07 < 7.32e-06 (0.105) 1 iFFT: n2=1.01e+00 ninf=1.51e+00 < 7.32e-06 (205712.585) 1 FAIL
pyopencl  R2C        (2105,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.24e-07 ninf=3.87e-07 < 7.32e-06 (0.053) 1 iFFT: n2=2.46e-07 ninf=5.37e-07 < 7.32e-06 (0.073) 1   OK
pyopencl  R2C        (2106,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=3.67e-07 ninf=3.86e-07 < 3.66e-06 (0.106) 1 iFFT: n2=3.06e-07 ninf=5.96e-07 < 3.66e-06 (0.163) 1   OK
pyopencl  R2C        (2107,) axes=      None ndim=   1    float32 lut=False inplace=0  norm=   1 C   FFT: n2=2.26e-07 ninf=2.51e-07 < 7.32e-06 (0.034) 1 iFFT: n2=1.91e-07 ninf=4.77e-07 < 7.32e-06 (0.065) 1   OK

Other tests seem OK so far, but there are some strangely failing with a timeout on the H100 for DCT4/DST4 1D transforms. What is interesting is that it is always the same lengths which fail (23100, 24024, 25920, 27300) if you look at two runs: http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-07-h100cu/pyvkfft-test.html http://ftp.esrf.fr/pub/scisoft/PyNX/pyvkfft-test/pyvkfft-test-2023-12-03-h100cu/pyvkfft-test.html

@vincefn ok, I found what I missed in the last fix about R2C on Apple, it should be correct now.

As for the H100, these sequences use >200KB of shared memory to be done in a single upload, and this territory is not really well tested - shared memory size has increased rapidly over the past two generations to the point that it matches the register file on H100 now. So I suspect here some weird register file spilling behavior can be observed. I limited the size of shared memory visible to VkFFT to 160KB for DCT/DST systems, this should solve the issue by switching to a multiple-uploads algorithm earlier.

@DTolm - great ! I can confirm all those failing tests now pass.

I'll restart the test suite on the different systems, we should be close to converging.

I'd like to add some information about the used FFT recipe used in a VkFFTApp, without probing too deep in the structures. Can you tell me if the following are correct:

After the VkFFTApp initialisation, app->configuration.allocateTempBuffer indicates that a buffer with the same size as the full array has been allocated as part of the VkFFTApp
To determine if a Rader or Bluestein transform has been used, either:
- if app->numRaderFFTPrimes>0, this indicates that a Rader transform will be performed (not sure how that applies to multiple axes with different lengths)
- does app->useBluesteinFFT[VKFFT_MAX_FFT_DIMENSIONS] indicate exclusively that the Bluestein algorithm (and not Rader) is used along each axis ?
can I use app->localFFTPlan->numAxisUploads[] to determine the number of uploads per axis ?

I tried looking at 1 and 2 above:

allocateTempBuffer seemed to be activated (with cuda) when arrays got large (around 3200), on the Apple M1 it seemed to be always 1
numRaderFFTPrimes also seemed to make sense according to what you told me in https://github.com/vincefn/pyvkfft/issues/25#issuecomment-1561312384 - there was just a case where this variable was still 0 for a length of 4418=24747

This is not essential (apart from the temp buffer allocation), but it can be useful for evaluation of performance.

@vincefn Sorry, it is hard to fix issues without having full access to the tested systems at all time, but I think I found the issue why fix didn't work. It is now also possible to limit the size of used shared memory as a parameter in configuration - maybe it can be useful.

As for your questions:

After the VkFFTApp initialisation, app->configuration.allocateTempBuffer indicates that a buffer with the same size as the full array has been allocated as part of the VkFFTApp

Yes, it indicates that VkFFT allocated a temporary buffer of size app->configuration.tempBufferSize[0]. Note that app->configuration.tempBufferSize[0] can be more than the system size in Bluestein algorithm or DCT/DST-I cases. On M1 there is only 32KB of shared memory, so the switch to multiple uploads happens earlier, but for single upload systems it should be off.

if app->numRaderFFTPrimes>0, this indicates that a Rader transform will be performed (not sure how that applies to multiple axes with different lengths)

This number indicates that FFT version of Rader transform will be used (which calculates Rader prime as a convolution theorem for P-1 length). 4418 uses a multiplication version of Rader algorithm, as 47 can't be done by FFT Rader (the requirement for it is that P-1 is decomposable a multiplication of primes up to 13) - 46 = 2 *23. The multiplication version of Rader algorithm calculates convolution by direct kernel multiplications and additions in O(P^2).

does app->useBluesteinFFT[VKFFT_MAX_FFT_DIMENSIONS] indicate exclusively that the Bluestein algorithm (and not Rader) is used along each axis ?

Yes, VkFFT doesn't use Rader's algorithm for Bluestein convolution calculation - if we pay the price to calculate the FFT with Bluestein's algorithm we can use any good sequences bigger than 2N-1. I guess the easiest way to distinguish the algorithms will be to use this flag and then check if primes >13 appear - they will use Rader's algorithm.

can I use app->localFFTPlan->numAxisUploads[] to determine the number of uploads per axis ?

Yes, unless the makeInversePlanOnly flag is enabled, then only localFFTPlan_inverse will be initialized.

The tests look good, they now take much longer with the extended DCT/DST testing.. I may have to find a way to shorten the test suites.

The H100 and A100 cuda tests finished without issue (there are just the 25 DCT4 failures on the A100, but we know this is solved by going from the 11.7 to the 12.2 cuda toolkit-I checked).

In the meantime I added a few functions to provide more information about the algorithms: this includes the size of the allocated buffer, the type of transform used ([r]adix, [R]ader or [B]luestein) and the number of uploads.

For example:

from pyvkfft.cuda import VkFFTApp
from pyvkfft.base import primes
import numpy as np
import pycuda.autoinit

app=VkFFTApp(shape=[400]*3,dtype=np.float32,inplace=True,dct=4) ; print(app)

> VkFFTApp[CUDA]:  (400,400,400)  DCT4/s/i [rrr] [111] buf=    0  

app=VkFFTApp(shape=[2050]*2,dtype=np.float32,inplace=True,dct=1) ; print(app)

> VkFFTApp[CUDA]:  (2050,2050)    DCT1/s/i [BB] [21] buf=128.8MB

The buffer size and the type of algorithm for each axis (rRB) will also be printed during systematic tests.

What I saw:

when a buffer is used for an R2C radix transform (e.g. cuda, 4200x4200 float32), the temp buffer is 2x the size of the real array
for DCT1:
- the temp buffer is 4x the array size for a 4200x4200 (cuda, float32, radix) transform
- using OpenCL, the buffer is 8x the array size using a 500x500x500 float32 array: 3.8GB buffer !

The DCT1 buffers are large - from the doc mapping on an C2C array of size 2N-2 (that explains a factor ~4x), and also in the 500x500x500 case the number of uploads (2,2,1) may play a role, but still 8x is a lot.

Anyway once more tests conclude positively I guess this will be ready for a new release :-)

The tests look good, they now take much longer with the extended DCT/DST testing.. I may have to find a way to shorten the test suites.

I guess this had to be done as the library is now almost feature-complete. The one thing missing is callbacks and while all the groundwork has been done for it (new R2R algorithms are done by using internal callbacks), the way how to interface it to the user in the multi-API environment is an open question.

using OpenCL, the buffer is 8x the array size using a 500x500x500 float32 array: 3.8GB buffer !

R->C increases memory x2, DCT-I for another x2, Bluestein's algorithm for another x2. Luckily, there is a need for only one temporary buffer.

Thanks for these extensive tests, they are really helpful!

using OpenCL, the buffer is 8x the array size using a 500x500x500 float32 array: 3.8GB buffer !

R->C increases memory x2, DCT-I for another x2, Bluestein's algorithm for another x2. Luckily, there is a need for only one temporary buffer.

I understand x4 (R->C + DCT1-), but why x8 in this case ? 500 does not use Bluestein.

Now the only main thing missing is testing half precision, but this would require a few tricks as there is little support in numpy (there is float16 but no complex32, so that'd require some on-the-fly conversion).

Oh - a lone error just appeared on the very slow AMD gfx900 testing: the 4420x4420 R2C forward (non-radix) transform fails (only the N_inf, so I guess a lone value is incorrect). I tested the inplace version, it also fails as the out-of-place..

500 does not use Bluestein.

DCT-I of length 500 is computed as C2C FFT of length 998= 2 * 499. 499-1=498 is divisible by 83, which forces Bluestein's algorithm to be used. So DCT-I of length 500 is computed as C2C FFT of length 2048 in the end.

I will check 4420 R2C error on AMD later today, thanks for finding it out!

Dear @DTolm , I don't know if you had time to look at the AMD 4420 error - it could be a singular error related to a driver issue, so I don't know if it's worth investigating extensively, as we could otherwise tag a new release version.

vincefn / pyvkfft

VkFFT now supports multi-upload R2R and R2C transform #32

Full test results