Encoding a video from images

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Encode a video using ffmpeg
2. Use the code provided below
3.

What is the expected output? What do you see instead?
An image shown every 0.5 seconds (2fps). Instead I see a long pause at the 
start showing the first image of the video, then every image is shown every 0.5 
second. (See attached video)

What version of the product are you using? On what operating system?
Android 4.0.3 / Windows 8.1

Please provide any additional information below.

File mediaStorageDir = new 
File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DCIM),
                "MyImages");

         File folder = mediaStorageDir;
            File[] listOfFiles = folder.listFiles();  
            if(listOfFiles.length>0)
            {       
           img = new IplImage[listOfFiles.length];      
           for (int j = 0; j < listOfFiles.length; j++) {       
             String files="";                
           if (listOfFiles[j].isFile()) 
           {
           files = listOfFiles[j].getName();
             }       
            String[] tokens = files.split("\\.(?=[^\\.]+$)");
        String name=tokens[0]; 

         System.out.println(" j " + name);

       img[j]=cvLoadImage("/sdcard/DCIM/MyImages/"+name+".jpg");
           }
            }

        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder("/sdcard/DCIM/MyImages/test.mp4",800,480);

            try {
               recorder.setVideoCodec(com.googlecode.javacv.cpp.avcodec.AV_CODEC_ID_MPEG4);
               recorder.setFormat("mp4");
               recorder.setFrameRate(2);
               recorder.setPixelFormat(com.googlecode.javacv.cpp.avutil.AV_PIX_FMT_YUV420P);
               recorder.start();

               int x = 0;
                 int y = 0;
                 for (int i=0;i< 200 && x<img.length;i++)

                   {

                     recorder.record(img[x]);

                    if (i>(y+10)) {
                        y=y+1;
                        x++;
                    }
                   }

               recorder.stop();
            }
            catch (Exception e){
               e.printStackTrace();
            }

Original issue reported on code.google.com by peter9...@googlemail.com on 18 Jan 2014 at 5:20

Attachments:

• [test (2).mp4](https://storage.googleapis.com/google-code-attachments/javacv/issue-412/comment-0/test %282%29.mp4)

[GoogleCodeExporter]

Which version of JavaCV does that happen with? Does it happen with another 
codec like AV_CODEC_ID_H264 as well?

Original comment by samuel.a...@gmail.com on 19 Jan 2014 at 7:02

[GoogleCodeExporter]

JavaCV version 0.7. Yes, same result with AV_CODEC_ID_H264.

Original comment by peter9...@googlemail.com on 19 Jan 2014 at 11:20

[GoogleCodeExporter]

Doing some research into this issue I found the following information: 'If you 
encounter problems, such as the first image is skipped or only shows for one 
frame, then use the fps video filter instead of -r for the output framerate'

Now the fps video filter uses the library libavfilter which I can see exists 
within javaCV but I can't find an fps option in there? 

http://ffmpeg.org/ffmpeg-filters.html#fps

Original comment by peter9...@googlemail.com on 21 Jan 2014 at 11:31

[GoogleCodeExporter]

We can achieve the same effect as that filter by calling record() on the same 
image multiple times. So, for example, you could try to record a video file at 
20 FPS, but call record() tens times on each image, effectively getting us 2 
FPS. I doubt that H.264 does not support 2 FPS, but it's possible that we have 
to tweak some of the options of x264 to get it working at such a low FPS...

Original comment by samuel.a...@gmail.com on 24 Jan 2014 at 8:09

[GoogleCodeExporter]

Ok so I call recorder.record(img[x]) 10 times in a row. The length of the video 
is correct but the first frame is still showing for longer than the others and 
has the effect of making the other frames show for less time. The attached 
video is encoded at 10fps while calling record() 10 times so should appear to 
be 1fps but it is not.

Original comment by peter9...@googlemail.com on 24 Jan 2014 at 10:53

Attachments:

• 20140124_104639.mp4

[GoogleCodeExporter]

Sam, if there's no immediate fix for this is it possible to set the PTS using 
the pts filter? Apparently it's faster/less lossy. It would therefore be better 
taking a video with a slowed down frame rate and speeding/slowing it as 
required using the pts.

https://trac.ffmpeg.org/wiki/How%20to%20speed%20up%20/%20slow%20down%20a%20video

Original comment by peter9...@googlemail.com on 27 Jan 2014 at 4:00

[GoogleCodeExporter]

I have not had the time to look at this yet, sorry about that.

I'm not sure if it's going to have the same effect, but we can set the PTS of 
each frame by calling `setFrameNumber()` or `setTimestamp()` (units are 
microseconds) before `record()`...

Original comment by samuel.a...@gmail.com on 28 Jan 2014 at 2:21

[GoogleCodeExporter]

No worries. I've been trying various solutions to no avail :(. 

I tried playing around with the timestamp, I was able to change the video speed 
but it was getting messy and nowhere near as simple as images to video.

Original comment by peter9...@googlemail.com on 28 Jan 2014 at 5:17

[GoogleCodeExporter]

Does this happen only on Android? I can't seem to reproduce the problem here on 
the desktop with Java SE. The attached Test class successfully produces the 
attached MP4 file showing numbers starting from 0 to 19, on the first frame up 
to the last 20th frame, respectively.

Original comment by samuel.a...@gmail.com on 2 Feb 2014 at 2:04

Attachments:

• Test.java
• images.mp4

[GoogleCodeExporter]

Thanks for the reply, I just tested your code on android. Works fine. I then 
played around with the code and managed to get it to work :D. Rather simply I 
messed around with the for loop. Cheers Sam. Code below for anyone else. 

 for (int i=0; i<img.length; i++)           
                       {
                         recorder.record(img[i]);

    publishProgress((int) ((i / (float) img.length) * 100));

                       }

Original comment by peter9...@googlemail.com on 2 Feb 2014 at 1:26

[GoogleCodeExporter]

Yes, I see now, that was a bug in your code... 

Original comment by samuel.a...@gmail.com on 2 Feb 2014 at 2:18

• Changed state: Done
• Added labels: Type-Other
• Removed labels: Type-Defect