TCP Issue

[achriki]

Hello, I'm working o a playback project and i'm using ONVIFClient class alongside with H264Transport to manage the RTSP recording and writing it to the h264 file. and i use ffmpeg to read the h264 file and pipe it to the browser. After i run some tests to display the playback i found the ffmpeg keep throwing the same error in every test :

Here is the code snippet that i use to write the recording data to h264 file :

   // Step 1: Create an RTSPClient instance

   const client = new ONVIFClient(createdDevice.username, createdDevice.password);

    // Step 2: Connect to a specified URL using the client instance

    const details = await client.connect(ReplayUri, { connection: "tcp" }); 
    console.log(`Stream. Codec is ` + details.codec);

    // Step 3: Open the output file
    if (details.codec == "H264") {
        // Step 4: Create H264Transport passing in the client, file, and details
        // This class subscribes to the 'data' event, looking for the video payload
        const playbackTestPath = path.join(process.cwd(),"playback.264");
        if(fs.existsSync(playbackContainerPath + ".h264")){
            fs.rmSync(playbackContainerPath + ".h264");
        }
        videoFile = fs.createWriteStream(playbackContainerPath + '.h264');
        const h264 = new H264Transport(client, videoFile,details);

        // call the playfrom function de 
        const streaming = await client.playFrom(from,to);

        const ffmpegSpawn = spawn('tail', [
            '-f', playbackContainerPath + ".h264"
        ]);
        // console.log(ffmpegSpawn.stdout)
        if(playbackSpawned){
            replayTailProcess.kill();  
        }
        // console.log(ffmpegSpawn)
        replayTailProcess = ffmpegSpawn;
        console.log("here we go")
        const ffmpegCommand = ffmpeg(ffmpegSpawn.stdout)
        .addInputOption("-fflags","nobuffer")
        .addInputOption("-probesize",32)
        .addInputOption("-analyzeduration",0)
        .on("start", function (cmdline) {
            playbackSpawned = true
            console.log("start processing")
        })
        .on("codecData", function () {
            //Camera online processing
        })
        .on("error", function (err) {
            console.log("An error occured: ", err.message);
        })
        .on("end", function () {
            //Treatment of camera disconnection
            console.log("end of stream");

        }).outputFormat("flv").addOutputOption("-preset", "ultrafast").videoCodec("libx264").noAudio()
        ffmpegReplayProcess = ffmpegCommand.pipe(stream)
    } else if(details.codec == "H265"){
        console.log("not yet supported")
    }

After some code review i went to check the change between my local server and the NVR and i found a big issue in the TCP acknowledgments, i saw that this there is an asynchrony between the RTP data and TCP [ACK] So NVR keep resending the same data waiting for TCP [ACK] that's explaining ffmpeg crash because there is just one frame who sent by NVR again and again.

But After this asynchrony operation the change RTP / TCP back to normal as it was in the beginning.

Please i want to know if this issue is related the yellowstone library or related to something else.

[RogerHardiman]

Hi I don't fully understand the issue report so I am making some guesses here.

I wonder if the difference on the Wireshark logs as that NodeJS is a single thread. When there RTP data is received, NodeJS will be writing to the file and I think the file write is taking place I assume NodeJS is not reading the TCP socket. NodeJS But in the top wireshark log, the Seq= number is incrementing. Re-sends at the TCP level would have the same Seq as that is the

As for ffmpeg, I don't know the cause of that error. As a test, if you let the .264 file finish being written, is ffmpeg able to read the whole file?

[achriki]

Hi Maybe the issue in NodeJS because his single threaded, but i tried to separate the steps into to steps, the first one is to let NodeJS fill the .264 file with the RTP data. and after that send it to ffmpeg to read the .264 file but i could achieve that because when NodeJS started writing the file he keep added the same RTP data to the file, and that make the file contained an invalid data, even after the logs get back to the normal state as i seed, the NodeJS is not going to stop writing the file it's like he didn't take the timestamp given in the playFrom function in consideration, so i forced the NodeJS server to stop running for not let the file consume a lot of storage. I think that's why ffmpeg display the error because he couldn't find any other frame to continue reading the file. If you have any ideas that could help to solve this issue i'll be thankful.

[RogerHardiman]

You could try this... Make ffmpeg read bytes from a socket. In yellowstone, when you get the RTP data, write it to the socket

[achriki]

Hi Thank you for the advice, I want to inform you that i fixed the timestamps issue in the playFrom function by calling the function before the instantiation of the H264Transport Object, and know NodeJs can finish writing the h264 file without crushing. But i found another issue when i tried to convert the h264 into mp4 using ffmpeg command. ffmpeg throw the Invalid NAL unit and No frame errors. it's seems to me that RTP Packet is not correct :

Note : if it's possible to give me the steps to bring the play back properly into my application maybe i arrange my code black in bad way Here is my code snippet :

If you have any ideas i'll be thankful.