Xilinx / Vitis-AI-Tutorials

MIT License
358 stars 144 forks source link

What is the correct pattern for CONV+BN layer #93

Open AjinkyaDeshpande39 opened 1 year ago

AjinkyaDeshpande39 commented 1 year ago

I have a CNN in built in pytorch

class Classifier(nn.Module):
    def __init__(self, resnetModel):
        super().__init__()
        self.resnet = resnetModel
        print(resnetModel.fc)
        self.fc_in_features = resnetModel.fc.in_features
        self.resnet.fc = nn.BatchNorm1d(num_features=self.fc_in_features)
        self.fc0 = nn.Linear(self.fc_in_features, 1024)
        # self.resnet.fc = self.fc0
        self.relu = nn.ReLU()
        self.fc1 = nn.Linear(1024,256)
        self.batchNorm2 = nn.BatchNorm1d(256)
        self.dropout1 = nn.Dropout(0.2)
        self.fc2 = nn.Linear(256,64)
        self.batchNorm3 = nn.BatchNorm1d(64)
        self.dropout2 = nn.Dropout(0.2)
        self.fc3 = nn.Linear(64,3)
        # self.softmax = nn.Softmax()
    def forward(self,x):
        x = self.resnet(x)
        x = self.fc0(x)
        x = self.relu(x)
        # x = self.dropout1(x)
        x = self.fc1(x)
        x = self.batchNorm2(x)
        x = self.relu(x)
        x = self.dropout1(x)
        x = self.fc2(x)
        x = self.batchNorm3(x)
        x = self.relu(x)
        x = self.dropout2(x)
        x = self.fc3(x)
        # x = self.softmax(x)
        return(x)

I quantize this model

quantizer = torch_quantizer(quant_mode, model, (rand_in), output_dir=quant_model, device=device) 
quantized_model = quantizer.quant_model

check test accuracy of both the quantized and non quantized model and then export the xmodel using following command

quantizer.export_xmodel(deploy_check=False, output_dir=quant_model)

The code runs properly and model gets quantized. The log file looks like this -

[VAIQ_NOTE][QUANTIZER_TORCH_NOT_FUSED_BN]: Node Classifier::Classifier/ResNet[resnet]/BatchNorm1d[fc]/9867 cannot be fused into CONV layers, this is not quantization friendly. It is recommended to adjsut the pattern to CONV+BN.

[VAIQ_NOTE][QUANTIZER_TORCH_NOT_FUSED_BN]: Node Classifier::Classifier/BatchNorm1d[batchNorm2]/input.311 cannot be fused into CONV layers, this is not quantization friendly. It is recommended to adjsut the pattern to CONV+BN.

[VAIQ_NOTE][QUANTIZER_TORCH_NOT_FUSED_BN]: Node Classifier::Classifier/BatchNorm1d[batchNorm3]/input.317 cannot be fused into CONV layers, this is not quantization friendly. It is recommended to adjsut the pattern to CONV+BN.

[VAIQ_NOTE]: =>Doing weights equalization...

[VAIQ_NOTE]: =>Quantizable module is generated.(./TCGA2/MODIFIED/ResNet50/Fold0/quant/Classifier.py)

[VAIQ_NOTE]: =>Get module with quantization.
[122, 134, 136]
non quantized model test_accuracy = 100.0 | test_loss =  0.005811731649406107
quantized model test_accuracy = 99.4898 | test_loss =  0.024748119462491072

[VAIQ_NOTE]: =>Converting to xmodel ...

[VAIQ_NOTE]: =>Successfully convert 'Classifier' to xmodel.(./TCGA2/MODIFIED/ResNet50/Fold0/quant/Classifier_int.xmodel)

My question is, here it says batch normalization layer cannot be used with conv layer and adjust the CONV+BN pattern. So what is this correct pattern ?