Open vvolhejn opened 2 years ago
@rino20 Could you take a look?
Hi @vvolhejn ,
Since you haven't set the pruning parameters, the default option is applied - ConstantSparsity, with pruning frequency 100. https://github.com/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/python/core/sparsity/keras/pruning_schedule.py#L141
That means, your model will be pruned at every 100 steps. Your example will run less than 100 steps if batchsize is larger than 2, so that's why you don't get pruned result (the training finishes before applying pruning)
Hope this helps,
Thank you for clearing this up. It makes sense explained like this, but it feels a bit unintuitive to me :/ The docs say "frequency: Only apply pruning every frequency
steps." which doesn't seem to imply the first pruning happens after frequency
steps.
Sorry for the confusion. We wished that "Only" in the sentence implies it, but that might not be enough.
Describe the bug When using
prune_low_magnitude()
, my model is not pruned if the batch size is low.System information
TensorFlow version (installed from source or binary): 2.8.0 installed via pip
TensorFlow Model Optimization version (installed from source or binary): 0.7.2 installed via pip
Python version: 3.9.10
Describe the expected behavior
model_for_pruning.fit
should sparsify the model independent of the batch size.Describe the current behavior
If the batch size is larger than 2 (this is the threshold in my example, at least), the network is not pruned.
Code to reproduce the issue
Based on the Pruning with Keras tutorial.
This prints:
So when the batch size is 1 or 2, everything works fine. But for anything larger, the model is not pruned.