search

usa-npn / cales-thermal-calendars

Estimating trends in phenology in the northeastern US
https://usa-npn.github.io/cales-thermal-calendars/spatial-trends-report.html
MIT License
0 stars 0 forks source link

Optimize BLAS/LAPACK/openMP #29

Closed Aariq closed 2 months ago

Aariq commented 2 months ago

Honestly, don't totally know what this means or what is involved, but want to double check my BLAS (or whatever the HPC is using) is using all cores. This will help GAMs scale better.

Aariq commented 2 months ago

Looks like R is using the macOS provided BLAS, which should be optimized, but it's not using the macOS LAPACK, and I might gain some performance by figuring out how to link to that. Tried some instructions I found and it broke R though.

Aariq commented 2 months ago

Ok, was able to follow the FAQ here to get a ~30x increase in matrix multiplication: https://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Which-BLAS-is-used-and-how-can-it-be-changed_003f

Not sure how to do something similar for LAPACK, but the improvements in BLAS might be enough!

Aariq commented 2 months ago

Next step is figuring out openMP which will allow me to use the ncv.threads argument in gam.control() https://mac.r-project.org/openmp/

Aariq commented 2 months ago

If I end up going with REML and bam() instead of NCV, then I can use a mirai cluster to speed things up with the cluster argument https://shikokuchuo.net/posts/21-mirai-parallel-clusters/

Aariq commented 2 months ago

Have to install mgcv from source and modify Makevars: https://stackoverflow.com/questions/78590558/how-can-i-enable-openmp-in-mac-os-for-mgcv-r-package https://mac.r-project.org/openmp/