Clip ingest from video file

[eacharya]

Objective

The current capture mechanism was designed with the view of capturing anything that can be played in a web browser. We need a simpler mechanism for ingesting videos that are already available as a stand-alone file.

Video is stored in Rasbari using the following model hierarchy: stream -> capture -> clip. Each clip is ~1 minute long and has an associated thumbnail. Capture has multiple clips that are collected in a sequential fashion. A stream has multiple captures that have been collected at different times.

Let us say that we want to run a detector model across 100 videos stored in youtube. In that scenario, we would:

1. Use youtube-dl to download the videos and store them in a folder
2. In the Rasbari UI, we would create a new stream
3. For each downloaded video, we would create a new capture object
4. Using ffmpeg, we would split a downloaded video into clips (with specific attributes) and associate these clips with the capture object for that video
5. Finally, along with the database entries, we would save the clips that are made to the storage server

This issue assumes 1 and 2 has been done and creates code for doing 3 through 5.

Coding suggestion

1. Follow the rake task code as a guide to create a new rake task. We can assume that we will be able to ingest the files from the Rasbari server rails environment.
2. As input, the rake task will take a stream ID and a folder with video files.
3. A ping step is required to ensure that storage server is alive and ready.
4. For each video, use ffmpeg to extract clips in a temporary folder and once ffmpeg exists, associate the clips with a new capture for the stream. Also, as new clip ids are created send clip file to storage server.

Rasbari server probably doesn't yet have ffmpeg - an apt install might be necessary.

For ffmpeg command, please refer to existing nimki code. Most likely, video->clip conversion command will be a variation of:

command = "ffmpeg -loglevel error -framerate 25 -video_size #{@width}x#{@height} -i :#{@input_filename} -c:v libx264 -pix_fmt yuv420p -crf 20 -preset veryfast -f segment -segment_time 60 -reset_timestamps 1 #{saveFolder}/#{clipId}.mp4"

FFMpeg website has great tutorials and it might be necessary to also use ffprobe as you write this code.

[DeepakZigvu]

@eacharya For the clips, how are length, frame_number_start, frame_number_end defined?

[eacharya]

I would suggest that we set:

length: 60 1000 = 60000 frame_number_start = 0 frame_number_end = 25 60 = 1500

During inference, we decode the video at a per-frame level. During that time, we can update these database entries.

On Mon, Jun 5, 2017 at 7:40 PM, DeepakZigvu notifications@github.com wrote:

@eacharya https://github.com/eacharya For the clips, how are length, frame_number_start, frame_number_end defined?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/zigvu/rasbari/issues/10#issuecomment-306365527, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOdgIGnTcXlHjKs4sY-hE3keJK5hZ_aks5sBLwMgaJpZM4NM1f8 .

[DeepakZigvu]

@eachary How is the split mp4 file associated with the clip? Video::Clip does not seem to have any value for the file. Is it based on frame start time relative to the first clip in 60 second interval?

[DeepakZigvu]

@eacharya Are the clips renamed based on id assigned by rails/MySQL in the folder streams/streamId/clipId/? Is created_at date value for clips used to find the order of the clips?

[eacharya]

For this rake task, here is the hierarchy of video:

• Stream: A customer's video dataset url here
• Capture: Each individual video url here
• Clip: The segmented clips

We create the hierarchy through database IDs.

Clip path names are according to stream id / capture id / clip id

[DeepakZigvu]

What is the mechanism to untar the file in the destination? I am sending the tar.gz file directly using storageClient.saveFile

[DeepakZigvu]

Rake task for ingesting clip is implemented. Command for calling rake task from rasbari folder:

rake ingest_clip:ingest stream_id=<ID> clip_file=<clip_file_location>

ID : ID of the stream clip_file_location: Location of the mp4 file.

[DeepakZigvu]

Downloading video using youtube-dl

youtube-dl -f mp4 <video-url>

-f mp4: format mp4

[DeepakZigvu]

@eacharya Output framerate of 25 specified for ffmpeg clip generation task

[arpgh]

Tested with >30min Youtube videos and clips are correctly being generated. One of the videos used is showing millisecond timer to test correct clip boundaries. Most difference I saw at these boundaries is 67ms, which is higher than 40ms expected for 25fps, but due to key frame positions, likely ok. Othewise, looking good so far.

[eacharya]

@DeepakZigvu - I made a few comments on the commit for this issue. There is some clean up required and bug we need to fix: https://github.com/zigvu/rasbari/commit/98b22e9701f4b68f6ef6b73e485d1c908217403

Let me know if you have any questions.