search

ethanhe42 / channel-pruning

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)
https://arxiv.org/abs/1707.06168
MIT License
1.07k stars 310 forks source link

Lasso Regression for Conv4_x required too much memory. #91

Closed mtyylx closed 6 years ago

mtyylx commented 6 years ago

Hi Yihui, When I'm trying to prune conv4_x layer using lasso regression, I encountered a memory problem.

For Conv4_2, consider 5000 images and 10 samples per image, and 512 input channel and 512 output channel, the Z matrix will be 48.8 GB in float32. And performing LASSO on such a big matrix requires 4x + more memory. How did you manage to calculate all this? Did your server has 128GB + memory? Or is there some way to workaround this? Thanks.

mtyylx commented 6 years ago

Got it. It's seem that in the original implementation, LASSO only fit on the resampled 400 patches of 50000 patches. So memory is no longer a problem.