tfjgeorge / nngeometry

{KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch
https://nngeometry.readthedocs.io
MIT License
203 stars 20 forks source link

adds test for KFC #75

Closed tfjgeorge closed 9 months ago

tfjgeorge commented 9 months ago

BUGFIX: KFAC for convolutions was scaled by the # of spatial positions but it should not.

actually the former implementation corresponds to what is derived in the KFC paper. However arguably the scaling by 1/S with S the number of spatial positions might not be optimal (see discussion in derivations/kfc.pdf) thus I instead scaled the test by S and reverted back to KFC expressions.

codecov[bot] commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d7225d9) 94.13% compared to head (ade2a1d) 94.89%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #75 +/- ## ========================================== + Coverage 94.13% 94.89% +0.75% ========================================== Files 31 31 Lines 3479 3464 -15 ========================================== + Hits 3275 3287 +12 + Misses 204 177 -27 ``` | [Flag](https://app.codecov.io/gh/tfjgeorge/nngeometry/pull/75/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Thomas+George) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/tfjgeorge/nngeometry/pull/75/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Thomas+George) | `94.89% <100.00%> (+0.75%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Thomas+George#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.