Closed zhujiem closed 1 year ago
I found that monarch_linear is a pure pytorch implementation, but some others are based on huggingface / trion backend. Which one is faster?
These are just various kinds of weight matrices that we've tried / played with over several projects.
Hi Tri. Thank you very much for your introduction! It is much more clear for me now. But it did not point out to pixelatedbutterfly linear layer. I even thought blocksparse_linear is for pixelfly. Could you also give me the quick link? I'd like to compare Monarch and Pixelfly.
Pixelfly is blocksparse_linear.py
with a specific sparsity pattern (FlatBlockButterflySparsityConfig).
You can check the config here to see an example.
Many thanks!
Hi, I am very interested in your work and want to try their applications. I have figured out the usage of "monarch_linear.py". But I am still confused about other layers with similar names.
Could you please briefly introduce them to help me better understand your code? Thx in advance.