Open niciwalter99 opened 2 years ago
@rino20 Could you take a look?
What if you look at the weights of the model after stripping the pruning with this?
Hi @YannPourcenoux
Since you haven't set the pruning parameters, the default option is applied - ConstantSparsity, with pruning frequency 100. https://github.com/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/python/core/sparsity/keras/pruning_schedule.py#L141
That means, your model will be pruned at every 100 steps. Your example will run less than 100 steps, (1000/64*2 < 100) so that's why you don't get pruned result (the training finishes before applying pruning)
When you have batchsize 16, it will over 100 steps (1000/16 *2 > 100) so it will be pruned well.
Hope this helps,
Okay that makes sense. But why does the 'frequency' argument exist at all in the ConstantSparsity function? It's useful in the PolynomialDecay function, but isn't it useless to have a frequency, when you just want a constant sparsity pruning and fine tune afterwards?
Describe the bug Prune Low Magnitude seems not to update the weights to 0 (I am using Constant Sparsity), when using a small dataset for training (1000 Images).
System information
TensorFlow version: 2.8.1
TensorFlow Model Optimization version (installed from source or binary): 0.7.2
Python version: 3.8.10
Describe the expected behavior Pruning should be working as normal on small dataset as it is working on bigger datasets.
Describe the current behavior The weights are not updated to 0 after a model_for_pruning.fit run (see in Code Example). The exact same example works if you increase the size of the dataset (var dataset_size) to 10000 or change the batch_size to 16. I don't think, that this is intended when using the Constant Sparsity Feature or am I doing something wrong here? Code to reproduce the issue