search

microsoft / MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
MIT License
5.8k stars 965 forks source link

AttributeError: 'NoneType' object has no attribute 'name' in FusedBatchNorm #925

Open machanic opened 3 years ago

machanic commented 3 years ago

Platform (like ubuntu 16.04/win10): ubuntu 16.04

Python version: 3.7.3

Source framework with version (like Tensorflow 1.4.1 with GPU): Tensorflow 1.4.0

Destination framework with version (like CNTK 2.3 with GPU): PyTorch 1.7.1

Pre-trained model path (webpath or webdisk path): https://drive.google.com/file/d/1awFM8y4A9jWcfUFaVs6S3CBGZramLBkY/view?usp=sharing Running scripts: mmconvert -sf tensorflow -in model-50.meta -iw model-50 --dstNode resnet10/logits/BiasAdd -df pytorch -om model-50.pth

machanic commented 3 years ago

It seems that this is a bug that caused by Fused Batch Normalization layer, but I did not use that layer.

I only use tf.layers.batch_normalization as the BN layer in ResNet-10, but I didn't pass fused argument, which is set to None by default. None means it uses a faster, fused implementation if possible

JiahaoYao commented 3 years ago

Hi @machanic , I looked into the network. Here is a temporal solution:

You might want to replace these two liners with 

        mean = self.get_parent(source_node.name, [3], True)
        var = self.get_parent(source_node.name, [4], True)

also, replace these two liners with 

            self.set_weight(source_node.name, 'mean', self.ckpt_data[mean.name.replace('Const', 'moving_mean')])

            self.set_weight(source_node.name, 'var', self.ckpt_data[var.name.replace('Const_1', 'moving_variance')])

You can find your src path via import mmdnn; print(mmdnn.__path__)

Most likely, after the temporal hotfixes, you can get 

    def forward(self, x):
        resnet10_Pad    = F.pad(x, (3, 3, 3, 3), mode = 'constant', value = 0)
        resnet10_conv1_Conv2D = self.resnet10_conv1_Conv2D(resnet10_Pad)
        resnet10_pool1_MaxPool, resnet10_pool1_MaxPool_idx = F.max_pool2d(resnet10_conv1_Conv2D, kernel_size=(3, 3), stride=(2, 2), padding=0, ceil_mode=False, return_indices=True)
        resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_pool1_MaxPool)
        resnet10_block1_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block1_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block1_unit_1_bottleneck_v2_Pad)
        resnet10_block1_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block1_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block1_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block1_unit_1_bottleneck_v2_add = resnet10_block1_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block1_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block1_unit_1_bottleneck_v2_add)
        resnet10_block2_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block2_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block2_unit_1_bottleneck_v2_Pad)
        resnet10_block2_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block2_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block2_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block2_unit_1_bottleneck_v2_add = resnet10_block2_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block2_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block2_unit_1_bottleneck_v2_add)
        resnet10_block3_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_Pad = F.pad(resnet10_block3_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1), mode = 'constant', value = 0)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block3_unit_1_bottleneck_v2_Pad)
        resnet10_block3_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block3_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block3_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block3_unit_1_bottleneck_v2_add = resnet10_block3_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block3_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm = self.resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm(resnet10_block3_unit_1_bottleneck_v2_add)
        resnet10_block4_unit_1_bottleneck_v2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_batch_normalization_FusedBatchNorm)
        resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D(resnet10_block4_unit_1_bottleneck_v2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_conv1_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv1_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad = F.pad(resnet10_block4_unit_1_bottleneck_v2_conv1_Relu, (1, 1, 1, 1))
        resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D_pad)
        resnet10_block4_unit_1_bottleneck_v2_conv2_Relu = F.relu(resnet10_block4_unit_1_bottleneck_v2_conv2_Conv2D)
        resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D = self.resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D(resnet10_block4_unit_1_bottleneck_v2_conv2_Relu)
        resnet10_block4_unit_1_bottleneck_v2_add = resnet10_block4_unit_1_bottleneck_v2_shortcut_Conv2D + resnet10_block4_unit_1_bottleneck_v2_conv3_Conv2D
        resnet10_batch_normalization_FusedBatchNorm = self.resnet10_batch_normalization_FusedBatchNorm(resnet10_block4_unit_1_bottleneck_v2_add)
        resnet10_Relu   = F.relu(resnet10_batch_normalization_FusedBatchNorm)
        resnet10_pool5  = torch.mean(resnet10_Relu, 3, True)
        resnet10_pool5  = torch.mean(resnet10_pool5, 2, True)
        resnet10_logits_Conv2D = self.resnet10_logits_Conv2D(resnet10_pool5)
        return resnet10_logits_Conv2D

Good Luck!

JiahaoYao commented 3 years ago

Hi @machanic, did this solve your issue?