non-deterministic wisdom output

[bmwiedemann]

While working on reproducible builds for openSUSE, I found that our postfish package varied between builds, because its postfish-wisdomrc varied. This is with fftw 3.3.10

To reproduce, use this minimal reproducer:

w="fftwf-wisdom -v -o postfish-wisdomrc rof128"
for i in $(seq 20) ; do $w >/dev/null 2>&1 ; md5sum postfish-wisdomrc ; done | sort | uniq -c
     18 499d3c0e49fe651f9af7dd1d3eeff879  postfish-wisdomrc
      2 4ef272e0fee6f14ee092086877af457d  postfish-wisdomrc

--- md5/499d3c0e49fe651f9af7dd1d3eeff879        2023-10-16 12:14:59.077228115 +0000
+++ md5/4ef272e0fee6f14ee092086877af457d        2023-10-16 12:14:58.747233965 +0000
@@ -1,6 +1,6 @@
 (fftw-3.3.10 fftwf_wisdom #x08ac4c16 #x457005cc #xea102cf7 #xd7ff9038
-  (fftwf_codelet_r2cfII_4 2 #x11048 #x11048 #x0 #x7df6d1cd #x84f9b02f #x28804c2c #x1df22293)
-  (fftwf_codelet_hc2cfdftv_4_sse2 0 #x11048 #x11048 #x0 #xfbe6070d #x75bc1578 #xf797496e #xf06a230d)
-  (fftwf_codelet_n2fv_32_sse2 0 #x11048 #x11048 #x0 #x6b0fa9b1 #xaf33c3b9 #x9e2f0af9 #x23d859ed)
-  (fftwf_codelet_r2cf_4 2 #x11048 #x11048 #x0 #x277111a9 #x5273d133 #x72fd5c22 #x23157970)
+  (fftwf_codelet_n2fv_16_sse2 0 #x11048 #x11048 #x0 #xe338bb01 #xcce8bf8f #x166215e4 #x2a142c67)
+  (fftwf_codelet_r2cfII_8 2 #x11048 #x11048 #x0 #xb4976b30 #x9902ac88 #xc9b16830 #x8e7ca7a4)
+  (fftwf_codelet_hc2cfdftv_8_sse2 0 #x11048 #x11048 #x0 #xfbe6070d #x75bc1578 #xf797496e #xf06a230d)
+  (fftwf_codelet_r2cf_8 2 #x11048 #x11048 #x0 #xd77870fb #xb3573fb2 #xf10c1820 #xc6197dde)
 )

I tried taskset 1 setarch -R fftw-wisdom ... but it still was non-deterministic.

Please ensure output only depends on inputs.

[bmwiedemann]

The full reproducer from postfish uses

w="fftwf-wisdom -v -o postfish-wisdomrc rif32 rof32 rib32 rob32 rif64 rof64 rib64 rob64 rif128 rof128 rib128 rob128 rif256 rof256 rib256 rob256 rif512 rof512 rib512 rob512 rif1024 rof1024 rib1024 rob1024 rif2048 rof2048 rib2048 rob2048 rif4096 rof4096 rib4096 rob4096 rif8192 rof8192 rib8192 rob8192 rif16384 rof16384 rib16384 rob16384"

please use that one to verify your fix.

[rdolbeau]

The fftw-wisdom binary is not expected to produce deterministic results, at least not in measure/patient/exhaustive (and there's little interest in having 'estimate' plans in the wisdom files). It only generates 'plans' for the specified transformations to enable deterministic reuse from those generated 'plans' afterward. It doesn't do more than the combination of 'fftw_plan()' and 'fftw_export_wisdom()'.

[stevengj]

Yes, it's not deterministic, because plan generation depends on runtime timing measurements. (fftw-wisdom defaults to using FFTW_PATIENT mode.) See the FFTW FAQ.

[bmwiedemann]

Thanks for that pointer to the FAQ. I can confirm that using fftwf-wisdom -e makes output deterministic.