empty tensor during inference?

Hey @SanniM3,
When I run inference using the pyscene fine tuned model, it crashes after about 5 videos with the following output. Any idea what might be causing this?
Evan
evan@mlpcw3-3:~/video_summarisation_git$ python -m generativeimage2text.vc_inference -p "{'type': 'multi_video_inference', 'videos_csv': 'data/test/pyscenedetect_frames/processed_data_test.csv', 'annotations_json_path': 'data/test/test_videodatainfo.json', 'model_path':'fine_tuned_models/pyscene_models/better_msrvtt_model_epoch1.pt', 'model_name':'GIT_BASE'}"
2023-03-15 22:12:44.005210: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64
2023-03-15 22:12:44.005410: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:/usr/local/nccl2/lib:/usr/local/cuda/extras/CUPTI/lib64
2023-03-15 22:12:44.005454: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-03-15 22:12:47,179.179 29284:vc_inference.py:341   <module>(): param:
{'annotations_json_path': 'data/test/test_videodatainfo.json',
 'model_name': 'GIT_BASE',
 'model_path': 'fine_tuned_models/pyscene_models/better_msrvtt_model_epoch1.pt',
 'type': 'multi_video_inference',
 'videos_csv': 'data/test/pyscenedetect_frames/processed_data_test.csv'}
/opt/conda/lib/python3.7/site-packages/torchvision/transforms/transforms.py:330: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum.
  "Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. "
2023-03-15 22:12:52,420.420 29284:decoder.py:833   __init__(): creating temperal embedding
2023-03-15 22:13:16,638.638 29284:vc_inference.py:217 multi_video_inference(): output: a man is driving a car
2023-03-15 22:13:37,928.928 29284:vc_inference.py:217 multi_video_inference(): output: a man talks about a car
2023-03-15 22:14:04,167.167 29284:vc_inference.py:217 multi_video_inference(): output: two men driving a car
2023-03-15 22:14:32,419.419 29284:vc_inference.py:217 multi_video_inference(): output: a man is talking about a motorcycle
2023-03-15 22:14:58,849.849 29284:vc_inference.py:217 multi_video_inference(): output: a girl is playing a video game
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/evan/video_summarisation_git/generativeimage2text/vc_inference.py", line 344, in <module>
    locals()[function_name](**kwargs)
  File "/home/evan/video_summarisation_git/generativeimage2text/vc_inference.py", line 214, in multi_video_inference
    'prefix': torch.tensor(input_ids).unsqueeze(0).cuda(),
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/evan/video_summarisation_git/generativeimage2text/layers/decoder.py", line 840, in forward
    result = self.forward_one(batch, return_info=False)
  File "/home/evan/video_summarisation_git/generativeimage2text/layers/decoder.py", line 852, in forward_one
    visual_features = torch.cat(features, dim=1)
RuntimeError: torch.cat(): expected a non-empty list of Tensors
SanniM3 / video_summarisation_git

empty tensor during inference? #17
