search

martinwholtmon / IT3920-2024-Master-MSIT

Master project for MSIT 2024 - Towards Efficient Human Action Recognition: The Role of Keyframe Selection in Video Processing
MIT License
0 stars 0 forks source link

X3D error #72

Closed martinwholtmon closed 2 months ago

martinwholtmon commented 2 months ago

Training on 1 epoch, 0 warmup I got this error: 

Traceback (most recent call last):
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/experiment.py", line 453, in <module>
    main(args)
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/experiment.py", line 428, in main
    trainer.fit(
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 544, in fit
    call._call_and_handle_interrupt(
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 580, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 987, in _run
    results = self._run_stage()
              ^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 1031, in _run_stage
    self._run_sanity_check()
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 1060, in _run_sanity_check
    val_loop.run()
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/loops/utilities.py", line 182, in _decorator
    return loop_run(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 135, in run
    self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 396, in _evaluation_step
    output = call._call_strategy_hook(trainer, hook_name, *step_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/trainer/call.py", line 309, in _call_strategy_hook
    output = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/pytorch_lightning/strategies/strategy.py", line 412, in validation_step
    return self.lightning_module.validation_step(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/src/har_project/trainer.py", line 86, in validation_step
    return self._shared_step(
           ^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/src/har_project/trainer.py", line 64, in _shared_step
    logits = self(features)
             ^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/src/har_project/trainer.py", line 53, in forward
    return self.model(x)
           ^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/git/IT3920-2024-Master-MSIT/src/har_project/models/x3d.py", line 45, in forward
    return self.model(x)
           ^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.cache/torch/hub/facebookresearch_pytorchvideo_main/pytorchvideo/models/net.py", line 43, in forward
    x = block(x)
        ^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.cache/torch/hub/facebookresearch_pytorchvideo_main/pytorchvideo/models/head.py", line 374, in forward
    x = self.pool(x)
        ^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.cache/torch/hub/facebookresearch_pytorchvideo_main/pytorchvideo/models/x3d.py", line 799, in forward
    x = self.pool(x)
        ^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cluster/home/martinwh/.conda/envs/torch2.2/lib/python3.12/site-packages/torch/nn/modules/pooling.py", line 717, in forward
    return F.avg_pool3d(input, self.kernel_size, self.stride,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: input image (T: 8 H: 6 W: 6) smaller than kernel size (kT: 13 kH: 5 kW: 5)

Not really sure what the issue is, MoViNet works fine.

martinwholtmon commented 2 months ago

I did figure this out. This means that we have to increase the input, either the number of frames or the resolution.