search

gallantlab / himalaya

Multiple-target linear models - CPU/GPU
https://gallantlab.github.io/himalaya
BSD 3-Clause "New" or "Revised" License
78 stars 13 forks source link

FIX only update batch cv_score on hypergrad model fitting early end #37

Closed cchen23 closed 2 years ago

cchen23 commented 2 years ago

In the previous version of the code, during hyper gradient descent if the change in deltas from iteration i to i+1 is small then model fitting for that batch ends early, and cv_scores is truncated to be i+1 x num_targets. However, if batching is used during model fitting then the cv_score truncation causes an error (The entire num_iters x num_targets matrix is truncated to i+1 x num_targets. This causes indexing issues because other batches may still need the full num_iters length matrix).

I changed the code to set cv_scores[i+1:, batch] to be cv_scores[i:, batch], since the batch deltas for iteration i+1 onwards are not updated, and therefore the cv_scores would not change for that batch after iteration i.

mvdoc commented 2 years ago

Can you add a test that makes the previous version of the code fail?

TomDLT commented 2 years ago

Good investigation, thanks! I added a non-regression test, I will merge after the tests.