Closed YoshikiKato0220 closed 3 months ago
This error doesn't occur in MCT v1.11.0. But in MCT v1.11.0, I have a #1055 error.
linear_programing.py
ipdb> aggr_ru [8400.0, 33600.0, 8400.0, 33600.0, 8400.0, 8400.0, 8400.0, 8400.0, 204800.0, 819200.0, 8400.0, 8400.0, 33600.0, 25600.0, 6400.0, 1600.0, 25600.0, 204800.0, 819200.0, 1600.0, 6400.0, 25600.0, 102400.0, 204800.0, 409600.0, 204800.0, 819200.0, 25600.0, 204800.0, 819200.0, 153600.0, 51200.0, 51200.0, 102400.0, 153600.0, 51200.0, 102400.0, 307200.0, 102400.0, 204800.0, 204800.0, 307200.0, 102400.0, 204800.0, 614400.0, 204800.0, 409600.0, 409600.0, 1228800.0, 819200.0, 409600.0, 307200.0, 102400.0, 204800.0, 204800.0, 614400.0, 409600.0, 204800.0, 204800.0, 51200.0, 102400.0, 153600.0, 51200.0, 51200.0, 51200.0, 102400.0, 102400.0, 409600.0, 102400.0, 102400.0, 102400.0, 102400.0, 102400.0, 204800.0, 204800.0, 204800.0, 819200.0, 204800.0, 204800.0, 204800.0, 204800.0, 204800.0, 409600.0, 409600.0, 819200.0, 819200.0, 1228800.0, 409600.0, 409600.0, 819200.0, 819200.0, 1638400.0, 3276800.0, 1228800.0, 102400.0, 409600.0, 102400.0, 409600.0, 102400.0, 204800.0, 102400.0, 102400.0, 204800.0, 409600.0, 409600.0, 819200.0, 1638400.0, 400.0, 1600.0, 6400.0, 25600.0, 6400.0, 1600.0, 25600.0, 102400.0, 409600.0, 25600.0, 102400.0, 409600.0, 25600.0, 102400.0, 409600.0, 25600.0, 102400.0, 409600.0] ipdb> p v 3276800.0
Hi @YoshikiKato0220 ,
According to the error message you are getting, It seems that you are trying to run mixed precision quantization to quantize the model's activations to a specific target memory size. The tutorial that you are running is not set to run this type of quantization, so I will need to know whether you made any changes to the code using the MCT, in order to figure out what could be the problem.
If you did try to run activation mixed precision, it is possible that the problem is that you provide the MCT with a memory restriction that is too low, such that the maximal activation tensor memory size can not be reduced to the specified target. If this is the case, I would suggest to try and provide a looser restriction on the activation memory size.
Let us know if this is helpful and if you need any other help with this issue.
Hi @ofirgo , Thanks for your comment, I only changed the code in terms of SKU-110K (not COCO).
Then I changed the code in accordance with your advise.
resource_utilization = mct.core.ResourceUtilization(resource_utilization_data.weights_memory 0.75) ↓ resource_utilization = mct.core.ResourceUtilization(resource_utilization_data.weights_memory 0.75, 3276800)
it worked. Thank you.
Could you tell me the effect when activation_memory is increased. Do the quantized model make less compression ratio or less accuracy ?
Hi @ofirgo , Thanks for your comment, I only changed the code in terms of SKU-110K (not COCO).
Then I changed the code in accordance with your advise.
resource_utilization = mct.core.ResourceUtilization(resource_utilization_data.weights_memory 0.75) ↓ resource_utilization = mct.core.ResourceUtilization(resource_utilization_data.weights_memory 0.75, 3276800)
it worked. Thank you.
Could you tell me the effect when activation_memory is increased. Do the quantized model make less compression ratio or less accuracy ?
I'm glad to hear this solves the issue for you. Regardless, I'm still not sure what caused the original issue, since you say you didn't change anything from the original tutorial code but the dataset. Do you know how to explain where the restriction of "1638400" activation memory came from in your original code that had the issue? Because, if you didn't provide it, then the expected behavior should be that the activation memory size is not restricted and you shouldn't have experienced the error.
What happened when you modified the resource_utilization call is that you added a restriction on the memory of the maximal activation tensor during model inference to 3276800. This is the size of the actual maximal activation tensor (according to your previous message), so this modification didn't affect the accuracy or the memory of the quantized model.
I'm keeping the issue open for now. I would appreciate it if you can provide more information regarding the way you originally called MCT (that causes the issue).
Thank you for raising this issue and for your help.
Hi @ofirgo I'm sorry to have troubled you. my code was wrong.
I had set resource_utilization_data not resource_utilization in keras_post_training_quantization's 3rd argment.
Issue Type
Bug
Source
source
MCT Version
2.0.0
OS Platform and Distribution
Ubuntu 18.04
Python version
3.10.14
Describe the issue
Expected behaviour
Code to reproduce the issue
Log output