`Traceback (most recent call last):
File "./tracking/train_ocean.py", line 259, in
main()
File "./tracking/train_ocean.py", line 250, in main
model, writer_dict = ocean_train(train_loader, model, optimizer, epoch + 1, curLR, config, writer_dict, logger, device=device)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/core/function.py", line 38, in ocean_train
cls_loss_ori, cls_loss_align, reg_loss = model(template, search, label_cls, reg_target=reg_label, reg_weight=reg_weight)
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, kwargs)
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], *kwargs[0])
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(input, kwargs)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/models/ocean.py", line 325, in forward
cls_align = self.align_head(reg_feature, offsets)
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, kwargs)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/models/connect.py", line 219, in forward
x = self.relu(self.rpn_conv(x, offset))
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, *kwargs)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/models/connect.py", line 203, in forward
x = self.conv(x, offset)
File "/home/dasima/anaconda3/envs/TracKit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(input, kwargs)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/models/dcn/deform_conv.py", line 237, in forward
self.dilation, self.groups, self.deformable_groups)
File "/home/dasima/pythonwork/TracKit/tracking/../lib/models/dcn/deform_conv.py", line 56, in forward
cur_im2col_step)
RuntimeError: shape '[1, 1, 2304]' is invalid for input of size 92160000
你好,我在修改Ocean.yaml的时候发现,将batch设为32或64能正常运行,如果继续调大(如128或192)则会出现如下错误
目测是deform_conv_cuda.cpp的问题,但对这一块我不知道怎么debug,请问如何解决这个问题。