Video Re-localization

by Yang Feng, Lin Ma, Wei Liu, Tong Zhang, Jiebo Luo

Introduction

Video Re-localization aims to accurately localize a segment in a reference video such that the segment semantically corresponds to a query video. For more details, please refer to our paper.

Citation

@InProceedings{feng2018video,
  author = {Feng, Yang and Ma, Lin and Liu, Wei and Zhang, Tong and Luo, 
  Jiebo},
  title = {Video Re-localization},
  booktitle = {ECCV},
  year = {2018}
}

Requirements

sudo apt install python-opencv
pip install tensorflow-gpu==1.13.1

Dataset

1. Download the ActivityNet features at link. You will get activitynet_v1-3.part-00 to activitynet_v1-3.part-05.

2. Merge and unzip the files. You'll get sub_activitynet_v1-3.c3d.hdf5.

   cat activitynet_v1-3.part-0* > temp.zip
   unzip temp.zip

3. Get the code and split the features.

   git clone https://github.com/fengyang0317/video_reloc.git
   cd video_reloc
   ln -s /path/to/sub_activitynet_v1-3.c3d.hdf5 data/
   python split_feat.py

4. [Optional] Download the all the videos into data/videos and get the number of frames in each video.

   python get_frame_num.py

5. Generate the dataset json.

   python create_dataset.py

   Model

6. Train the model.

   python match.py --data_dir data

7. Eval the model. The results may be slightly different from the values reported in the paper.

   python eval.py --data_dir data --ckpt_path saving/model.ckpt-(best val ckpt)