This (more precisely, commit e11edc75e330be205a6a28b619eaeb4f11b7a2f7) reduces the peak memory usage of kfold_varsel() massively by avoiding that the output of select() and get_submodls() from all CV folds is stored in common (very large) objects (search_path_cv and submodls_cv). This movement of large portions of code should also be beneficial for a future parallelization across CV folds (because with this PR, we have all code that needs to be run on one worker within a single function) and in my opinion, it also enhances the readability of kfold_varsel().
The readability of kfold_varsel()-related functions is also enhanced by commits 23b274ef11b7b2932f0d537332c06d37c6bf5352 to c64ac577a32053947910d3233d49869abd7b72ec which are just refactoring commits.
This (more precisely, commit e11edc75e330be205a6a28b619eaeb4f11b7a2f7) reduces the peak memory usage of
kfold_varsel()massively by avoiding that the output ofselect()andget_submodls()from all CV folds is stored in common (very large) objects (search_path_cvandsubmodls_cv). This movement of large portions of code should also be beneficial for a future parallelization across CV folds (because with this PR, we have all code that needs to be run on one worker within a single function) and in my opinion, it also enhances the readability ofkfold_varsel().The readability of
kfold_varsel()-related functions is also enhanced by commits 23b274ef11b7b2932f0d537332c06d37c6bf5352 to c64ac577a32053947910d3233d49869abd7b72ec which are just refactoring commits.