search

Tencent / PocketFlow

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
https://pocketflow.github.io
Other
2.78k stars 490 forks source link

why weight_sparsification can reduce flops #217

Closed smalltingting closed 5 years ago

smalltingting commented 5 years ago

hello，may I ask you two questions?

why weight_sparsification can reduce flops？

and will the size of model reduce after weight_sparsification ？

yuanyuanli85 commented 5 years ago

If you are using standard tensorflow or hardware, they can't take the advantage of sparse model to reduce model size and flops. But, will give chance to particular hardware or library to skip the computing with zero weights. let's say i have a GPU with is designed for sparse computing, then the sparse model will run much faster on that than standard GPU. I think there will be more and more platform/hardware can support that in feature.

jiaxiang-wu commented 5 years ago

Thanks for the detailed explanation from @yuanyuanli85 . Some extra information:

  1. Theoretically, FLOPs can be reduced, if you can take advantage of the sparsity during computation. However, most inference engine do not have highly-optimized implementation for sparse matrix/tensor operations, and sparse matrices are often converted into dense ones so the FLOPs does not change.
  2. In *.ckpt models, zero-valued weights are still stored, hence there won't be any changes in the model size.
smalltingting commented 5 years ago

If you are using standard tensorflow or hardware, they can't take the advantage of sparse model to reduce model size and flops. But, will give chance to particular hardware or library to skip the computing with zero weights. let's say i have a GPU with is designed for sparse computing, then the sparse model will run much faster on that than standard GPU. I think there will be more and more platform/hardware can support that in feature.

Thanks a lot.