Recording video with wrong duration

[1120198]

What would you like to submit?

• [x] question
• [ ] bug report
• [ ] feature request

Issue description

I was trying to update Accord 3.8.0.0 to 3.8.2.0-alpha but I was having some issues when recording video, while in the previous version everything worked fine.

Basically when I record a 10 sec video, in the file properties and video player the video has 10 seconds, but when playing it, it takes way longer to finish (around 22 seconds)… The video is slow down.

After a lot of tweaking in the settings I gave up and started looking for similar problems in google in order to find out a solution but haven't found anything useful.

After some time I came across with this project https://github.com/cesarsouza/screencast-capture/ from @cesarsouza which implements Accord version 3.8.2.0-alpha. I downloaded a zip file with the source, compile it, fixed a compilation error by changing a Dispose() to Stop() and got it building correctly, but when running the application the same issue happened.

The main methods of the above indicated project are:

public void StartRecording()
{
    if (IsRecording || !IsPlaying)
        return;

    Rectangle area = CaptureRegion;
    string fileName = NewFileName();

    int height = area.Height;
    int width = area.Width;
    Rational framerate = new Rational(1000, screenStream.FrameInterval);
    int videoBitRate = 1200 * 1000;

    OutputPath = Path.Combine(main.CurrentDirectory, fileName);
    RecordingStartTime = DateTime.MinValue;
    videoWriter = new VideoFileWriter();
    videoWriter.BitRate = videoBitRate;
    videoWriter.FrameRate = framerate;
    videoWriter.Width = width;
    videoWriter.Height = height;
    videoWriter.VideoCodec = VideoCodec.H264;
    videoWriter.VideoOptions["crf"] = "18"; // visually lossless
    videoWriter.VideoOptions["preset"] = "veryfast";
    videoWriter.VideoOptions["tune"] = "zerolatency";
    videoWriter.VideoOptions["x264opts"] = "no-mbtree:sliced-threads:sync-lookahead=0";

    videoWriter.Open(OutputPath);

    HasRecorded = false;
    IsRecording = true;
}

void VideoPlayer_NewFrameReceived(object sender, Accord.Video.NewFrameEventArgs eventArgs)
{
    DateTime currentFrameTime = eventArgs.CaptureFinished;

    // Encode the last frame at the same time we prepare the new one
    Task.WaitAll(
        Task.Run(() =>
        {
            lock (syncObj) // Save the frame to the video file.
            {
                if (IsRecording)
                {
                    if (RecordingStartTime == DateTime.MinValue)
                        RecordingStartTime = DateTime.Now;

                    TimeSpan timestamp = currentFrameTime - RecordingStartTime;
                    if (timestamp > TimeSpan.Zero)
                        videoWriter.WriteVideoFrame(this.lastFrame, timestamp, this.lastFrameRegion);
                }
            }
        }),

        Task.Run(() =>
        {
            // Adjust the window according to the current capture
            // mode. Also adjusts to keep even widths and heights.
            CaptureRegion = AdjustWindow();

            // Crop the image if the mode requires it
            if (CaptureMode == CaptureRegionOption.Fixed ||
                CaptureMode == CaptureRegionOption.Window)
            {
                crop.Rectangle = CaptureRegion;

                eventArgs.Frame = croppedImage = crop.Apply(eventArgs.Frame, croppedImage);
                eventArgs.FrameSize = crop.Rectangle.Size;
            }

            //// Draw extra information on the screen
            bool captureMouse = Settings.Default.CaptureMouse;
            bool captureClick = Settings.Default.CaptureClick;
            bool captureKeys = Settings.Default.CaptureKeys;

            if (captureMouse || captureClick || captureKeys)
            {
                cursorCapture.CaptureRegion = CaptureRegion;
                clickCapture.CaptureRegion = CaptureRegion;
                keyCapture.Font = Settings.Default.KeyboardFont;

                using (Graphics g = Graphics.FromImage(eventArgs.Frame))
                {
                    g.CompositingQuality = CompositingQuality.HighSpeed;
                    g.SmoothingMode = SmoothingMode.HighSpeed;

                    float invWidth = 1; // / widthScale;
                    float invHeight = 1; // / heightScale;

                    if (captureMouse)
                        cursorCapture.Draw(g, invWidth, invHeight);

                    if (captureClick)
                        clickCapture.Draw(g, invWidth, invHeight);

                    if (captureKeys)
                        keyCapture.Draw(g, invWidth, invHeight);
                }
            }
        })
    );

    // Save the just processed frame and mark 
    // it to be encoded in the next iteration:
    lastFrame = eventArgs.Frame.Copy(lastFrame);
    //lastFrameTime = currentFrameTime;
    lastFrameRegion = new Rectangle(0, 0, eventArgs.FrameSize.Width, eventArgs.Frame.Height);
}

Is there any clear issue that I am not seeing?

Thank you.

[1120198]

I was analysing the console output I have verified that in the following logs:

pts:4.032000e+003   pts_time:0.252  dts:2.016000e+003   dts_time:0.126  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:6.720000e+003   pts_time:0.42   dts:3.360000e+003   dts_time:0.21   duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:1.075200e+004   pts_time:0.672  dts:5.376000e+003   dts_time:0.336  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:1.344000e+004   pts_time:0.84   dts:6.720000e+003   dts_time:0.42   duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:1.612800e+004   pts_time:1.008  dts:8.064000e+003   dts_time:0.504  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:1.881600e+004   pts_time:1.176  dts:9.408000e+003   dts_time:0.588  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:2.150400e+004   pts_time:1.344  dts:1.075200e+004   dts_time:0.672  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:2.553600e+004   pts_time:1.596  dts:1.276800e+004   dts_time:0.798  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:2.822400e+004   pts_time:1.764  dts:1.411200e+004   dts_time:0.882  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:3.091200e+004   pts_time:1.932  dts:1.545600e+004   dts_time:0.966  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:3.494400e+004   pts_time:2.184  dts:1.747200e+004   dts_time:1.092  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:3.897600e+004   pts_time:2.436  dts:1.948800e+004   dts_time:1.218  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:4.166400e+004   pts_time:2.604  dts:2.083200e+004   dts_time:1.302  duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:4.704000e+004   pts_time:2.94   dts:2.352000e+004   dts_time:1.47   duration:6.720000e+002  duration_time:0.042 stream_index:0
pts:5.107200e+004   pts_time:3.192  dts:2.553600e+004   dts_time:1.596  duration:6.720000e+002  duration_time:0.042 stream_index:0

The PTS is always the double of DTS. That explains why the video is played in a slow down way… Unfortunately I have no idea why this is happening.

Anyone has any clue?

[Schnjan]

I have the exact same problem with all codecs.

When trying to provide an adjusted (lower) timestamp to avoid the problem, the 'WriteVideoFrame' function will throw an ArgumentException. Version 3.8.0 works as expected, but I really like the 3.8.2 VideoOptions dictionary.

Any help is very appreciated.

[jfrank14]

Any luck on this? I'm finding that WriteVideoFrame(frame) works but WriteVideoFrame(frame, timestamp) always throws an exception, and I can't figure out why. But you need the timestamp in order to write audio frames and have them line up correctly.

[XenonLX]

Same issue here, pts_time is always double of dts_time and as result video is twice longer then player detects. Started reproducing on 3.8.2-alpha - doesn't reproduce on 3.8.0. As a video source, I've used the MJPEG stream which doesn't have audio at all. Tried both overloads of WriteVideoFrame method with a timestamp and without - works the same.

[MercurialForge]

I've come upon the same issue. Because I need the videooptions property this puts in me one hell of a bind. I've been searching for a day or more on how to fix this issue, but I've had no luck.

I tried updating the ffmpeg build dlls (3.2.2 => 3.2.4) but it had no effect. This leads me to believe it's an issue with the accord source code. I'll be looking into that next but I'm out of my league in C++.

If anyone can help it would be greatly appreciated.

[XenonLX]

@MercurialForge https://github.com/accord-net/framework/issues/1713 this may be the issue source.

[MercurialForge]

@XenonLX you're the real hero of time. That was it. I fixed it in the source and recompiled the .dll and videos produce as expected. Thank you very much for pointing out that documented issue!

[cakirbey]

@MercurialForge , @XenonLX ; I'm having the same problem. How did you solve it? Can you share the dll you recompile? Where can I download? Thanks in advance.

[XenonLX]

PM me your email, I'll send you my compiled dlls.

On Mon, Dec 23, 2019, 09:36 cakirbey notifications@github.com wrote:

@MercurialForge https://github.com/MercurialForge , @XenonLX https://github.com/XenonLX ; I'm having the same problem. How did you solve it? Can you share the dll you recompile? Where can I download? Thanks in advance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/accord-net/framework/issues/1474?email_source=notifications&email_token=ACXUZSYCSSRWTQT47T553ITQ2BTBDA5CNFSM4FKOZ5CKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHQOPGY#issuecomment-568387483, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXUZS4GC3O5LIN6BBTOQMLQ2BTBDANCNFSM4FKOZ5CA .

[cakirbey]

Firstly, thank you for your interest @XenonLX , I couldn't figure out how to send a private message from here. my email is: jomolocco@gmail.com . thanks again.

[cakirbey]

Hi again, I used DLL which is the fix sent by @XenonLX and "The PTS is always the double of DTS." bug was solved. But there is still almost twice the difference in the time taken during recording and the video duration. In the 122 second (02.02 min) period, 50 seconds of video recording is created. I couldn't solve it. I am using VP8 Codec. The code I use is as follows.

Record Start: videoCodec = VideoCodec.Vp8; bitrate : 1250000 framerate :10 height :720 width: 1280 videoSource.VideoResolution: 1280x720, 30 fps (30 max fps), 24 bpp FileWriter.BitRate = bitRate; FileWriter.FrameRate = new Accord.Math.Rational((int)framerate ); FileWriter.Width = width; FileWriter.Height = height; FileWriter.VideoCodec = videoCodec;
FileWriter.Open(FileName); _recording = true; StartTimeStamp = DateTime.Now;

newFrame Event:

private void VideoSourcePlayer1_NewFrame(object sender, ref Bitmap image)
  {
         if (_recording)
                {
                    try
                    {
                        Bitmap CurrentBitmap = image;
                        DateTime CurrentTimeStamp = DateTime.Now;
                        TimeSpan CurrentTimeSpan = StartTimeStamp - CurrentTimeStamp;
                        Console.WriteLine("Video Write Timespan: " + CurrentTimeSpan.ToString());
                        FileWriter.WriteVideoFrame(CurrentBitmap, CurrentTimeSpan);

                       // FileWriter.WriteVideoFrame(image);  
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine("VideoSourcePlayer1_NewFrame catch: " + ex.Message);
                    }
                }
}

Little Part of the debug output:

Video Write Timespan: -00:01:55.4107365 pts:4,730000e+004 pts_time:47,3 dts:4,730000e+004 dts_time:47,3 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:55.6136263 pts:4,740000e+004 pts_time:47,4 dts:4,740000e+004 dts_time:47,4 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:55.8354960 pts:4,750000e+004 pts_time:47,5 dts:4,750000e+004 dts_time:47,5 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:56.5161088 pts:4,760000e+004 pts_time:47,6 dts:4,760000e+004 dts_time:47,6 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:56.7309841 pts:4,770000e+004 pts_time:47,7 dts:4,770000e+004 dts_time:47,7 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:56.9758413 pts:4,780000e+004 pts_time:47,8 dts:4,780000e+004 dts_time:47,8 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:57.1947200 pts:4,790000e+004 pts_time:47,9 dts:4,790000e+004 dts_time:47,9 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:57.4375834 pts:4,800000e+004 pts_time:48 dts:4,800000e+004 dts_time:48 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:01:57.5804983

The last two outputs:

Video Write Timespan: -00:02:02.7505463 pts:5,000000e+004 pts_time:50 dts:5,000000e+004 dts_time:50 duration:1,000000e+002 duration_time:0,1 stream_index:0 Video Write Timespan: -00:02:02.9884038 pts:5,010000e+004 pts_time:50,1 dts:5,010000e+004 dts_time:50,1 duration:1,000000e+002 duration_time:0,1 stream_index:0

Why is there a difference between timespan and dts/pts time? Could the problem be due to this? Note: It is the same issue with timespan or without timespan writing.

[cakirbey]

Note: When I set the framerate to 30; 95 seconds (1.35 min) recording time is 16 seconds in the video.

As the frame rate increases, the length of the video gets shorter and shorter than the actual value it should be.

Output: Video Write Timespan: -00:01:35.1086178 pts:1,606700e+004 pts_time: 16,067 dts:1,606700e+004 dts_time: 16,067 duration:3,300000e+001 duration_time:0,033 stream_index:0 Video Write Timespan: -00:01:35.2565302 pts:1,610000e+004 pts_time: 16,1 dts:1,610000e+004 dts_time: 16,1 duration:3,300000e+001 duration_time:0,033 stream_index:0 Video Write Timespan: -00:01:35.3984494 pts:1,613300e+004 pts_time: 16,133 dts:1,613300e+004 dts_time: 16,133 duration:3,300000e+001 duration_time:0,033 stream_index:0

So the video duration is like dts_time. Why is the video duration not value of timespan time, I still can't find it.