$ python tools/train.py --id fc --caption_model newfc --input_json data/cocotalk.json --input_fc_dir data/cocotalk_fc --input_att_dir data/cocotalk_att --input_label_h5 data/cocotalk_label.h5 --batch_size 10 --learning_rate 5e-4 --learning_rate_decay_start 0 --scheduled_sampling_start 0 --checkpoint_path log_fc --save_checkpoint_every 6000 --val_images_use 5000 --max_epochs 30
DataLoader loading json file: data/cocotalk.json
vocab size is 9487
DataLoader loading h5 file: data/cocotalk_fc data/cocotalk_att data/cocotalk_box data/cocotalk_label.h5
max sequence length in data is 16
read 123287 image features
assigned 113287 images to split train
assigned 5000 images to split val
assigned 5000 images to split test
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
/home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice.
fc_feat = att_feat.mean(0)
Read data: 0.0012667179107666016
Save ckpt on exception ...
model saved to log_fc/model.pth
Save ckpt done.
Traceback (most recent call last):
File "tools/train.py", line 183, in train
model_out = dp_lw_model(fc_feats, att_feats, labels, masks, att_masks, data['gts'], torch.arange(0, len(data['gts'])), sc_flag, struc_flag, drop_worst_flag)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, kwargs)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], *kwargs[0])
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(input, kwargs)
File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/modules/loss_wrapper.py", line 47, in forward
loss = self.crit(self.model(fc_feats, att_feats, labels[..., :-1], attmasks), labels[..., 1:], masks[..., 1:], reduction=reduction)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/CaptionModel.py", line 33, in forward
return getattr(self, ''+mode)(*args, *kwargs)
File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 136, in _forward
p_fc_feats, p_att_feats, pp_att_feats, p_att_masks = self._prepare_feature(fc_feats, att_feats, att_masks)
File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 943, in _prepare_feature
fc_feats = self.fc_embed(fc_feats)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(input, **kwargs)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1370, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: size mismatch, m1: [10 x 0], m2: [2048 x 512] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290
if self.use_fc:
try:
fc_feat = self.fc_loader.get(str(self.info['images'][ix]['id']))
except:
# Use average of attention when there is no fc provided (For bottomup feature)
fc_feat = att_feat.mean(0)//这是dataloader的第290行
else:
fc_feat = np.zeros((0), dtype='float32')
$ python tools/train.py --id fc --caption_model newfc --input_json data/cocotalk.json --input_fc_dir data/cocotalk_fc --input_att_dir data/cocotalk_att --input_label_h5 data/cocotalk_label.h5 --batch_size 10 --learning_rate 5e-4 --learning_rate_decay_start 0 --scheduled_sampling_start 0 --checkpoint_path log_fc --save_checkpoint_every 6000 --val_images_use 5000 --max_epochs 30 DataLoader loading json file: data/cocotalk.json vocab size is 9487 DataLoader loading h5 file: data/cocotalk_fc data/cocotalk_att data/cocotalk_box data/cocotalk_label.h5 max sequence length in data is 16 read 123287 image features assigned 113287 images to split train assigned 5000 images to split val assigned 5000 images to split test /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) Read data: 0.0012667179107666016 Save ckpt on exception ... model saved to log_fc/model.pth Save ckpt done. Traceback (most recent call last): File "tools/train.py", line 183, in train model_out = dp_lw_model(fc_feats, att_feats, labels, masks, att_masks, data['gts'], torch.arange(0, len(data['gts'])), sc_flag, struc_flag, drop_worst_flag) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, kwargs) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], *kwargs[0]) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/modules/loss_wrapper.py", line 47, in forward loss = self.crit(self.model(fc_feats, att_feats, labels[..., :-1], attmasks), labels[..., 1:], masks[..., 1:], reduction=reduction) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, **kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/CaptionModel.py", line 33, in forward return getattr(self, ''+mode)(*args, *kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 136, in _forward p_fc_feats, p_att_feats, pp_att_feats, p_att_masks = self._prepare_feature(fc_feats, att_feats, att_masks) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 943, in _prepare_feature fc_feats = self.fc_embed(fc_feats) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, **kwargs) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward return F.linear(input, self.weight, self.bias) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1370, in linear ret = torch.addmm(bias, input, weight.t()) RuntimeError: size mismatch, m1: [10 x 0], m2: [2048 x 512] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290
这与我没有预先下载bottomup feature的f30kbu.att.pth有关系吗