search

liuziwei7 / voxel-flow

Video Frame Synthesis using Deep Voxel Flow (ICCV 2017 Oral)
https://liuziwei7.github.io/projects/VoxelFlow
215 stars 50 forks source link

dataset & network #2

Open AshinCarlXiao opened 6 years ago

AshinCarlXiao commented 6 years ago

I am trying to reproduce your work lately, and I got three questions.

  1. how you divide the train/test videos from UCF101 dataset, is there a file name list of train/test video available?
  2. how you extract triple frames from video? Is there any criterion that the triple frames should suffice? In your paper, your explain "We sampled frame triplets with obvious motion". how your define the "obvious motion"? Is that possible to provide the code for making triple frames dataset from UCF101 videos?
  3. the network you used in your paper seem like u-net akin structure. but in your code, it seem like you use different net structure that don't use skipping connect. Is my understanding right?

very looking forward to your reply, thanks a lot!!!

sesse commented 6 years ago

Regarding point 3, it seems also that the loss function in the code is different from what's in the paper. There may be other differences; I haven't gone into it in detail yet.

linchuming commented 6 years ago

@AshinCarlXiao @sesse The big problem is the work use the tanh function as activate function in the last layer. It is not reasonable. The flow value will be restricted in [-1.0, 1.0].

liuziwei7 commented 6 years ago

This repository only contains the basic network setup for deep voxel flow. Skip connections and multi-scale architecture are not included here. For more updates, please refer to our newly released pytorch-voxel-flow: https://github.com/lxx1991/pytorch-voxel-flow .

Note that the voxel flow representation implemented in the network is actually normalized flow ranging from [-1, 1]. When applied to real frames, they have been re-scaled to the full size as: flow = height (or width) * flow. Please refer to the code ("utils/geo_layer_utils.py") for more details.