wenwei202 / iss-rnns

Sparse Recurrent Neural Networks -- Pruning Connections and Hidden Sizes (TensorFlow)
Apache License 2.0
73 stars 21 forks source link

add_blockwise_grouplasso: raise NotImplementedError? #1

Closed YikSanChan closed 6 years ago

YikSanChan commented 6 years ago

Hi,

I have read the ISS paper on arxiv, and fail to understand the code. In add_blockwise_grouplasso, It raises NotImplementedError('Not debugged. And the implementation is very slow when block is small.')

But the zero out part inside the method is definitely "ISS" in the paper. Does it mean, even though the code is slow, you still currently use this method to produce the result in the paper?

Thank you.

YikSanChan commented 6 years ago

And in this line, https://github.com/wenwei202/iss-rnns/blob/5c8f497ada4d87bc379c285374b78fbd0ab0074b/ptb/ptb_word_lm.py#L377, it actually use structure lasso instead of group lasso.

Still the same question as above, thank you!

wenwei202 commented 6 years ago

@YikSanChan structure lasso is one example of group lasso. In the code, I call it structure lasso because we can learn the ISS by removing one group of weights. Please ignore add_blockwise_grouplasso, which is never used for the paper.