Frame-Stream is not working as expected

piercus commented 2 years ago

On branch https://github.com/alezanai/kvs-parser/tree/frame-stream

@dharmik-dalwadi-seaflux please edit this issue to add detailed explanations

Actual

@dharmik-dalwadi-seaflux please explain what is the actual bahavior

Our Scenario is we extracting the frame objects using beamcoder in frame-stream.js class.
But when we are try to encode the frame objects using beamcoder in using kvs-to-frames.js we are getting this type of error. which is saying current pts is less than previous pts.

Error

Writing file tmp/frame-0.jpeg
Writing file tmp/frame-1.jpeg
Writing file tmp/frame-2.jpeg
Writing file tmp/frame-3.jpeg
[mjpeg @ 0x3780d80] Invalid pts (65849178822160) <= last (65849178822200)
[mjpeg @ 0x3780d80] Invalid pts (65849178822160) <= last (65849178822200)
Writing file tmp/frame-4.jpeg
(node:23634) UnhandledPromiseRejectionWarning: Error: TypeError: Cannot read property 'data' of undefined
    at /home/dharmik/Documents/Projects/klap/kvs-parser/scripts/kvs-to-frames.js:39:15
(node:23634) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:23634) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Assertion !avpkt->data || avpkt->data != avctx->internal->byte_buffer failed at src/libavcodec/encode.c:45
Aborted (core dumped)

Expected

@dharmik-dalwadi-seaflux please explain what is the expected bahavior

It should encode all the 129 frames and save it into the jpeg files.

dharmik-dalwadi-seaflux commented 2 years ago

@piercus, I see that this issue is not occuring when i are try to create new becmcoder object for each frame like this.

stream.on('data', frame => {
    const encoder = beamcoder.encoder({
        name: 'mjpeg',
        width: frame.width,
        height: frame.height,
        pix_fmt: 'yuvj420p',
        time_base: [1, 1],
    });

    encoder.encode(frame).then(jpeg => {
        const filename = `tmp/frame-${count}.jpeg`;
        console.log(`Writing file ${filename}`);
        fs.writeFileSync(filename, jpeg.packets[0].data);
        count++;
    }).catch(error => {
        throw new Error(error);
    });
});

piercus commented 2 years ago

@dharmik-dalwadi-seaflux can you please open a PR on this branch (do NOT mix the branches) including the fix.

Please write a script in ./scripts which is using FrameStream + jpg to save all images in the ./tmp folder and make sure all the images are visible.

I will review this once ready

dharmik-dalwadi-seaflux commented 2 years ago

@piercus I have created the PR can you please review it? https://github.com/alezanai/kvs-parser/pull/15

piercus commented 2 years ago

@dharmik-dalwadi-seaflux i'm experiencing different problems while running node scripts/kvs-to-frames.js

received SIGSEGV

➜  kvs-parser git:(DHA-frame-stream-issue) ✗ node scripts/kvs-to-frames.js
Aerostat Beam Coder  Copyright (C) 2019  Streampunk Media Ltd
GPL v3.0 or later license. This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions. Conditions and warranty at:
https://github.com/Streampunk/beamcoder/blob/master/LICENSE
Using FFmpeg version 4.3.2-0york0~18.04
Writing file tmp/frame-00000.jpg
Writing file tmp/frame-00001.jpg
Writing file tmp/frame-00002.jpg
Writing file tmp/frame-00003.jpg
Writing file tmp/frame-00004.jpg
Writing file tmp/frame-00005.jpg
...
Writing file tmp/frame-00106.jpg
PID 23885 received SIGSEGV for address: 0x6e086
[1]    23885 segmentation fault (core dumped)  node scripts/kvs-to-frames.js

order of frames in strange

Please pull the code from DHA-frame-stream-issue and then run

ffmpeg -i tmp/frame-%05d.jpg tmp/video.mp4

the result is here

You can see that the video is shaking a lot, i don't know why but this is strange. Please create one sub issue for each of those problem, with actual/expected issue format we will discuss them separately.

In each of the issue, tell me if you are able to reproduce the issue on your side

alezanai / kvs-parser