KeyError when converting resnet18 from pytorch to caffe

[zengchang233]

Thank you for your nice work! I encoutered some problems when converting resnet18 from pytorch to caffe. I just modified the example/resnet_pytorch_2_caffe.py to convert pretrained model as following

import sys
sys.path.insert(0,'.')
import torch
from torch.autograd import Variable
from torchvision.models import resnet
import pytorch_to_caffe

if __name__=='__main__':
    name='resnet18'
    resnet18=resnet.resnet18(pretrained=True)
    #checkpoint = torch.load("/home/shining/Downloads/resnet18-5c106cde.pth")

    #resnet18.load_state_dict(checkpoint)
    resnet18.eval()
    input=torch.ones([1,3,224,224])
    pytorch_to_caffe.trans_net(resnet18,input,name)
    pytorch_to_caffe.save_prototxt('{}.prototxt'.format(name))
    pytorch_to_caffe.save_caffemodel('{}.caffemodel'.format(name))

But I got the KeyError prompt like following.

Add blob       add_blob8       : torch.Size([1, 512, 7, 7])
140475289706360:batch_norm_blob20 getting
140475289705856:relu_blob15 getting
layer4.1.relu
140475289706216:add_blob8 getting
relu17 was added to layers
140475289706216:relu_blob17 was added to blobs
Add blob      relu_blob17      : torch.Size([1, 512, 7, 7])
140475289706216:relu_blob17 getting
view1 was added to layers
140475191369800:view_blob1 was added to blobs
Add blob       view_blob1      : torch.Size([1, 512])
Traceback (most recent call last):
  File "example/resnet_pytorch_2_caffe.py", line 16, in <module>
    pytorch_to_caffe.trans_net(resnet18,input,name)
  File "./pytorch_to_caffe.py", line 612, in trans_net
    out = net.forward(input_var)
  File "/root/anaconda3/envs/torch10/lib/python3.6/site-packages/torchvision/models/resnet.py", line 161, in forward
    x = x.view(x.size(0), -1)
  File "./pytorch_to_caffe.py", line 410, in _view
    bottom=[log.blobs(input)],top=top_blobs)
  File "./pytorch_to_caffe.py", line 88, in blobs
    print("{}:{} getting".format(var, self._blobs[var]))
  File "./pytorch_to_caffe.py", line 31, in __getitem__
    return self.data[key]
KeyError: 140475289706432

I guess maybe there are some bugs when convert torch.Tensor.view method to caffe.

May you check it later? Thanks a lot.

[LilySys]

you could first include

from torch.nn.modules.utils import _list_with_default

and then add

def _adaptive_avg_pool2d(raw, input, output_size):

_output_size = _list_with_default(output_size, input.size())

x = raw(input, _output_size)

_pool('ave', raw, input, x, input.shape[2], input.shape[2], 0, False)

return x 

and

F.adaptive_avg_pool2d=Rp(F.adaptive_avg_pool2d, _adaptive_avg_pool2d)

in pytorch_to_caffe.py.

[ghost]

@LilySys code solves the error in the pooling layer. However, it looks like current model available through pip, uses 'reshape' layer instead of 'view'. (The latest torchvision model uses 'flatten' layer, though.)

You should add a handler for reshape layer like the following:

def _reshape(input, *args):
    x=raw_reshape(input, *args)
    if not NET_INITTED:
        return x     layer_name=log.add_layer(name='reshape')
    top_blobs=log.add_blobs([x],name='reshape_blob')
    layer=caffe_net.Layer_param(name=layer_name,type='Reshape', bottom=[log.blobs(input)],top=top_blobs)
    dims=list(args)
    dims[0]=0 # the first dim should be batch_size
    layer.param.reshape_param.shape.CopyFrom(caffe_net.pb.BlobShape(dim=dims))
    log.cnet.add_layer(layer)
    return x

and

try:
    raw_view=Variable.view
    Variable.view=_view
    raw_reshape=Variable.reshape
    Variable.reshape=_reshape

and

except:
    # for new version 0.4.0 and later version
    for t in [torch.Tensor]:
        raw_view = t.view
        t.view = _view
        raw_reshape = t.reshape
        t.reshape = _reshape

[yutian11308023]

@Jungho-Jo ,hi ,I try your method, but it can not work. when I convert the flatten layer it can not come into the _reshape(), How should I do?

[lcaikk1314]

@Jungho-Jo ,hi ,I try your method, but it can not work. when I convert the flatten layer it can not come into the _reshape(), How should I do?

Hi ,have you solved the problem？I meet the same problem.

[OPPOA113]

met the same problem