search

UARK-AICV / VLTinT

[AAAI 2023 Oral] VLTinT: Visual-Linguistic Transformer-in-Transformer for Coherent Video Paragraph Captioning
https://uark-aicv.github.io/VLTinT/
65 stars 6 forks source link

YouCook2 #13

Open mrzhang66 opened 11 months ago

mrzhang66 commented 11 months ago

Hello, thank you very much for your outstanding work. Could you please share the feature files of the YouCook2 dataset, similar to the structure of the ActivityNet dataset?

mrzhang66 commented 11 months ago

Because I encountered a "Traceback (most recent call last)" issue while running the following commands: python tools/run_net.py --cfg configs/Kinetics/SLOWONLY_8x8_R50.yaml --feature_extraction --num_features 100 --video_dir path/to/dir/rescaled --feat_dir path/to/data/[anet/yc2]/c3d_env TEST.CHECKPOINT_FILE_PATH models/SLOWONLY_8x8_R50.pkl NUM_GPUS 1 TEST.CHECKPOINT_TYPE caffe2 TEST.BATCH_SIZE 1 DATA.SAMPLING_RATE 1 DATA.NUM_FRAMES 16 DATA_LOADER.NUM_WORKERS 0 and python tools/bbox_extract.py path/to/dir/rescaled path/to/dir/bbox --config-file configs/COCO-Detection/faster_rcnn_R_101_FPN_3x.yaml --sampling-rate 16 --target-frames 100 --opts MODEL.WEIGHTS model_final_f6e8b1.pkl I received the following error: Traceback (most recent call last): File "/public2/home/yangxingr/SlowFast-extract_featmap/tools/run_net.py", line 224, in main() File "/public2/home/yangxingr/SlowFast-extract_featmap/tools/run_net.py", line 169, in main feature_extract( File "/public2/home/yangxingr/SlowFast-extract_featmap/tools/feature_extraction.py", line 250, in feature_extract video_extraction_loader = loader.construct_loader(cfg, path_to_video) File "/public/home/yangxingr/.conda/envs/detectron2/lib/python3.9/site-packages/slowfast-1.0-py3.9.egg/slowfast/datasets/loader.py", line 103, in construct_loader File "/public/home/yangxingr/.conda/envs/detectron2/lib/python3.9/site-packages/slowfast-1.0-py3.9.egg/slowfast/datasets/build.py", line 31, in build_dataset File "/public/home/yangxingr/.conda/envs/detectron2/lib/python3.9/site-packages/slowfast-1.0-py3.9.egg/slowfast/datasets/video.py", line 85, in init RuntimeError: step must be nonzero and Traceback (most recent call last): File "/public2/home/yangxingr/detectron/detectron2-master/tools/bbox_extract.py", line 129, in for idx, frame_preds in bbox_extractor.run_on_video(video, num_frames, frames_per_second): File "/public2/home/yangxingr/detectron/detectron2-master/tools/bbox_extractor.py", line 58, in run_on_video sampling_pts = torch.arange( RuntimeError: step must be nonzero, but I am sure that the video can be played normally. However, I am certain that the video can be played normally.

yangxingrui commented 11 months ago

Same Problem

yangxingrui commented 11 months ago

If the video path is "path/to/dir/rescaled," an error will occur due to OpenCV reading a frame count and frame rate equal to 0. However, if the video path is "raw_videos," it can run without issues. Why?

yangxingrui commented 11 months ago

Sorry, I found that the rescaled files generated by rerunning the command python preprocess/rescale_video.py --video-root path/to/dir/*.mp4 --output-root path/to/dir/rescaled --frame-dir path/to/dir/middle_frames can be read by cv2. It seems like this issue is an unexpected occurrence.