search

calgaryml / condensed-sparsity

[ICLR 2024] Dynamic Sparse Training with Structured Sparsity
https://openreview.net/forum?id=kOBkxFRKTA
MIT License
17 stars 3 forks source link

Sparse init #38

Closed mklasby closed 1 year ago

mklasby commented 1 year ago

Sparse initalization methods added -> Kaiming normal, kaiming uniform, and "torch" init modified for sparse fan-ins.