search

MCG-NJU / DDM

[CVPR 2022] Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection
MIT License
49 stars 3 forks source link

The size of tensor a (10) must match the size of tensor b (11) #8

Closed GUNDAM1638 closed 1 year ago

GUNDAM1638 commented 1 year ago

I used the video __NrybzYzUg_000415_000425.mp4 and followed guide.md to prepare data Ran test.py with the Namespace(batch_size=128, data_dir='', dataset='kinetics_multiframes', model='multiframes_resnet', no_resume_opt=False, num_classes=2, pred_output='./multif-pred_outputs', rank=0, resume='../checkpoint.pth.tar', train_split='train', val_split='val') Got the error and didn't know the reason Traceback (most recent call last): File "D:\DFL_BASE\DDM-main\DDM-Net\test.py", line 162, in main() File "D:\DFLBASE\DDM-main\DDM-Net\test.py", line 115, in main outps, , _ = model(inps.cuda(non_blocking=True)) File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\parallel\data_parallel.py", line 159, in forward return self.module(*inputs[0], *kwargs[0]) File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\resnetGEBD.py", line 670, in forward intra_rgb_feat = self.intra_transformer1(x4, pos)[-1].permute(0, 2, 1) File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 67, in forward tgt, src, memory_key_padding_mask=None, pos=pos_embed, query_pos=query_embed File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(*input, *kwargs) File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 123, in forward query_pos=query_pos, File "C:\ProgramData\Anaconda3\envs\DDM\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 300, in forward query_pos, File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 222, in forward_post key=self.with_pos_embed(memory, pos), File "D:\DFL_BASE\DDM-main\DDM-Net\modeling\transformer.py", line 185, in with_pos_embed return tensor if pos is None else tensor + pos RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 0

Thanks

GUNDAM1638 commented 1 year ago

Here is my conda env

packages in environment at C:\ProgramData\Anaconda3\envs\DDM:

#

Name Version Build Channel

backcall 0.2.0 pyhd3eb1b0_0 defaults blas 2.20 mkl defaults ca-certificates 2023.01.10 haa95532_0 defaults certifi 2022.12.7 py37haa95532_0 defaults colorama 0.4.6 py37haa95532_0 defaults cudatoolkit 11.0.221 h74a9793_0 defaults decorator 5.1.1 pyhd3eb1b0_0 defaults einops 0.6.0 pyhd8ed1ab_0 defaults flit-core 3.6.0 pyhd3eb1b0_0 defaults freetype 2.12.1 ha860e81_0 defaults intel-openmp 2022.1.0 h59b6b97_3788 defaults ipdb 0.13.11 pyhd8ed1ab_0 defaults ipython 7.31.1 py37haa95532_1 defaults jedi 0.18.1 py37haa95532_1 defaults jpeg 9b hb83a4c4_2 defaults lerc 3.0 hd77b12b_0 defaults libblas 3.8.0 20_mkl defaults libcblas 3.8.0 20_mkl defaults libdeflate 1.8 h2bbff1b_5 defaults liblapack 3.8.0 20_mkl defaults liblapacke 3.8.0 20_mkl defaults libpng 1.6.37 h2a8f88b_0 defaults libtiff 4.5.0 h8a3f274_0 defaults libuv 1.40.0 he774522_0 defaults libwebp 1.2.4 h2bbff1b_0 defaults libwebp-base 1.2.4 h2bbff1b_0 defaults lz4-c 1.9.4 h2bbff1b_0 defaults m2w64-gcc-libgfortran 5.3.0 6 defaults m2w64-gcc-libs 5.3.0 7 defaults m2w64-gcc-libs-core 5.3.0 7 defaults m2w64-gmp 6.1.0 2 defaults m2w64-libwinpthread-git 5.0.0.4634.697f757 2 defaults matplotlib-inline 0.1.6 py37haa95532_0 defaults mkl 2020.2 256 defaults msys2-conda-epoch 20160418 1 defaults ninja 1.10.2 haa95532_5 defaults ninja-base 1.10.2 h6d14046_5 defaults numpy 1.21.6 py37h2830a78_0 defaults opencv-python 4.6.0.66 pypi_0 pypi openssl 1.1.1s h2bbff1b_0 defaults pandas 1.2.3 py37h08fd248_0 defaults parso 0.8.3 pyhd3eb1b0_0 defaults pathlib 1.0.1 pypi_0 pypi pickleshare 0.7.5 pyhd3eb1b0_1003 defaults pillow 9.3.0 py37hdc2b20a_1 defaults pip 22.3.1 py37haa95532_0 defaults prompt-toolkit 3.0.36 py37haa95532_0 defaults pygments 2.11.2 pyhd3eb1b0_0 defaults python 3.7.16 h6244533_0 defaults python-dateutil 2.8.2 pyhd3eb1b0_0 defaults python_abi 3.7 2_cp37m defaults pytorch 1.7.1 py3.7_cuda110_cudnn8_0 pytorch pytz 2022.7 py37haa95532_0 defaults setuptools 59.8.0 py37h03978a9_1 defaults six 1.16.0 pyhd3eb1b0_1 defaults sqlite 3.40.1 h2bbff1b_0 defaults tk 8.6.12 h2bbff1b_0 defaults toml 0.10.2 pyhd3eb1b0_0 defaults torchaudio 0.7.2 py37 pytorch torchvision 0.8.2 py37_cu110 pytorch tqdm 4.64.1 py37haa95532_0 defaults traitlets 5.7.1 py37haa95532_0 defaults typing_extensions 4.4.0 py37haa95532_0 defaults vc 14.2 h21ff451_1 defaults vs2015_runtime 14.27.29016 h5e58377_2 defaults wcwidth 0.2.5 pyhd3eb1b0_0 defaults wheel 0.37.1 pyhd3eb1b0_0 defaults wincertstore 0.2 py37haa95532_2 defaults xz 5.2.10 h8cc25b3_1 defaults zlib 1.2.13 h8cc25b3_0 defaults zstd 1.5.2 h19a0ad4_0 defaults

gulucaptain commented 1 year ago

I analyze the reason of this issue is the original checkpoint is been replaced. This happen due to the difference between the input frame number and the position embedding. When you generate the "pickle" through the generate_pickle.py, modify the "shift = np.arange(-frame_per_side, frame_per_side)" with "shift = np.arange(-frame_per_side, frame_per_side+1)" in Line 111. Then, this issue can be solved.

JackyTown commented 1 year ago

I analyze the reason of this issue is the original checkpoint is been replaced. This happen due to the difference between the input frame number and the position embedding. When you generate the "pickle" through the generate_pickle.py, modify the "shift = np.arange(-frame_per_side, frame_per_side)" with "shift = np.arange(-frame_per_side, frame_per_side+1)" in Line 111. Then, this issue can be solved.

Yes, and comment 'shift[shift >= 0] += 1' in the next line.