search

desihub / specex

DESI spectrograph PSF fitting
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

mkl optional #55

Closed marcelo-alvarez closed 3 years ago

marcelo-alvarez commented 3 years ago

Requires environment variable DESI_SPX_MKL to be set at build time in order to use the MKL implementation of BLAS. If DESI_SPX_MKL is not set (default behaviour) then the OpenBLAS implementation will be used.

This change avoids segmentation faults on otherwise successful jobs on KNL nodes at nersc that caused srun to return a non-zero result on the current master version, which uses the MKL implmenetation of BLAS by default.

No change in performance is found in this branch with respect to master on a single KNL node (i.e. mean PSF fit times of 340 sec and 343 sec, respectively) using the following command:

export OMP_NUM_THREADS=4
srun -N 1 -n 41 -c 4 desi_proc   --traceshift --cameras r14 -n 20211010 -e 103615  --mpi

and the relative difference in the PSF fits, diff, is zero to all digits of precision at wavelengths = [6000, 6700, 7300], where diff is defined as follows:

psf1=readpsf(args.psf1)
psf2=readpsf(args.psf2)
xx1,yy1,fpix1=psf1.xypix(args.fiber,args.wavelength)
xx2,yy2,fpix2=psf2.xypix(args.fiber,args.wavelength)
diff = np.abs(np.sum(fpix1*fpix2)/np.sum(fpix2**2)-1)

Please review and merge if appropriate.

julienguy commented 3 years ago

Great you found the issue. I trust you it's working and I merge because the changes are minimal.