ruotianluo / ImageCaptioning.pytorch

I decide to sync up this repo and self-critical.pytorch. (The old master is in old master branch for archive)
MIT License
1.43k stars 409 forks source link

训练时报错,我使用的是flickr30数据集 #156

Open wensenwang opened 2 years ago

wensenwang commented 2 years ago

$ python tools/train.py --id fc --caption_model newfc --input_json data/cocotalk.json --input_fc_dir data/cocotalk_fc --input_att_dir data/cocotalk_att --input_label_h5 data/cocotalk_label.h5 --batch_size 10 --learning_rate 5e-4 --learning_rate_decay_start 0 --scheduled_sampling_start 0 --checkpoint_path log_fc --save_checkpoint_every 6000 --val_images_use 5000 --max_epochs 30 DataLoader loading json file: data/cocotalk.json vocab size is 9487 DataLoader loading h5 file: data/cocotalk_fc data/cocotalk_att data/cocotalk_box data/cocotalk_label.h5 max sequence length in data is 16 read 123287 image features assigned 113287 images to split train assigned 5000 images to split val assigned 5000 images to split test /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) /home/yl/下载/ImageCaptioning.pytorch-master/captioning/data/dataloader.py:290: RuntimeWarning: Mean of empty slice. fc_feat = att_feat.mean(0) Read data: 0.0012667179107666016 Save ckpt on exception ... model saved to log_fc/model.pth Save ckpt done. Traceback (most recent call last): File "tools/train.py", line 183, in train model_out = dp_lw_model(fc_feats, att_feats, labels, masks, att_masks, data['gts'], torch.arange(0, len(data['gts'])), sc_flag, struc_flag, drop_worst_flag) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, kwargs) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], *kwargs[0]) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/modules/loss_wrapper.py", line 47, in forward loss = self.crit(self.model(fc_feats, att_feats, labels[..., :-1], attmasks), labels[..., 1:], masks[..., 1:], reduction=reduction) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, **kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/CaptionModel.py", line 33, in forward return getattr(self, ''+mode)(*args, *kwargs) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 136, in _forward p_fc_feats, p_att_feats, pp_att_feats, p_att_masks = self._prepare_feature(fc_feats, att_feats, att_masks) File "/home/yl/下载/ImageCaptioning.pytorch-master/captioning/models/AttModel.py", line 943, in _prepare_feature fc_feats = self.fc_embed(fc_feats) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, **kwargs) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward return F.linear(input, self.weight, self.bias) File "/home/yl/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1370, in linear ret = torch.addmm(bias, input, weight.t()) RuntimeError: size mismatch, m1: [10 x 0], m2: [2048 x 512] at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:290

    if self.use_fc:
        try:
            fc_feat = self.fc_loader.get(str(self.info['images'][ix]['id']))
        except:
            # Use average of attention when there is no fc provided (For bottomup feature)
            fc_feat = att_feat.mean(0)//这是dataloader的第290行
    else:
        fc_feat = np.zeros((0), dtype='float32')

这与我没有预先下载bottomup feature的f30kbu.att.pth有关系吗