Bad RTSP data framing

[oleaasbo]

Hello!

I ran a 15 min test of this library and got this error:

Stream Test: New WebSocket Connection (1 total)
./node_modules/yellowstone/dist/RTSPClient.js:425 speed=0.886x    
                throw new Error("Bug in RTSP data framing, please file an issue with the author with stacktrace.");
                ^

Error: Bug in RTSP data framing, please file an issue with the author with stacktrace.
    at RTSPClient._onData (./node_modules/yellowstone/dist/RTSPClient.js:425:23)
    at Socket.emit (events.js:197:13)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:145:17)

I can do more tests as I have integrated this library into a project that I will work a lot more on. Tell me what I can do to help solve this issue.

Btw. When the error occurred my computer had locked the screen(because i did not use the computer). When I unlocked I noticed the error. I ran a longer test while using the computer and no error occurred.

[RogerHardiman]

I will explain what the error means and then we can work out what is happening.

You are using RTSP in the mode where it mixes RTSP messages (which are the high level commands like DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN) with the low level RTP data which is the video and audio data.

The data stream from the video server uses special markers in the data stream to identify if the next bit of data is an RTSP message or if it is Video data or Audio data (ie the RTP data)

What the error means is that Yellowstone read some data and was unable to decide of the data was a RTSP message or an RTP message.

Now you may get the error if it fails to read any data.

If your computer locked, maybe the Socket connection to the video server stopped and that may cause the error.

[oleaasbo]

I got this error again without a screen lock. This time it took 1 hour before the error occurred. I would like to keep the stream going after this error has happened. What will happen if i remove "throw error" and replace it with console.log? Will it keep streaming and ignore the bad frame or whatever it is? :)

{"error":{},"level":"error","message":"uncaughtException: Bug in RTSP data framing, please file an issue with the author with stacktrace.\nError: Bug in RTSP data framing, please file an issue with the author with stacktrace.\n    at RTSPClient._onData (/node_modules/yellowstone/dist/RTSPClient.js:428:23)\n    at Socket.emit (events.js:197:13)\n    at addChunk (_stream_readable.js:288:12)\n    at readableAddChunk (_stream_readable.js:269:11)\n    at Socket.Readable.push (_stream_readable.js:224:10)\n    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:145:17)","stack":"Error: Bug in RTSP data framing, please file an issue with the author with stacktrace.\n    at RTSPClient._onData (/node_modules/yellowstone/dist/RTSPClient.js:428:23)\n    at Socket.emit (events.js:197:13)\n    at addChunk (_stream_readable.js:288:12)\n    at readableAddChunk (_stream_readable.js:269:11)\n    at Socket.Readable.push (_stream_readable.js:224:10)\n    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:145:17)","exception":true,"date":"Wed Feb 27 2019 14:13:44 GMT+0100 (GMT+01:00)","process":{"pid":22129,"uid":1000,"gid":1000,"cwd":"./","execPath":"/usr/bin/node","version":"v11.10.0","argv":["/usr/bin/node","/index.js"],"memoryUsage":{"rss":95731712,"heapTotal":57884672,"heapUsed":32662184,"external":20386569}},"os":{"loadavg":[1.1298828125,1.46826171875,1.671875],"uptime":794645},"trace":[{"column":23,"file":"/node_modules/yellowstone/dist/RTSPClient.js","function":"RTSPClient._onData","line":428,"method":"_onData","native":false},{"column":13,"file":"events.js","function":"Socket.emit","line":197,"method":"emit","native":false},{"column":12,"file":"_stream_readable.js","function":"addChunk","line":288,"method":null,"native":false},{"column":11,"file":"_stream_readable.js","function":"readableAddChunk","line":269,"method":null,"native":false},{"column":10,"file":"_stream_readable.js","function":"Socket.Readable.push","line":224,"method":"push","native":false},{"column":17,"file":"internal/stream_base_commons.js","function":"TCP.onStreamRead [as onread]","line":145,"method":"onStreamRead [as onread]","native":false}]}

[RogerHardiman]

Hi, If you remove the 'throw' then the library will not be able to recover properly and the best option is to stop the RTSP stream and reconnect. Yellowstone could try and read all the following bytes until it sees the $ sign or a RTSP message string, but there is no guarantee that the next $ is the start of the next block of RTP data or just a a byte with the value of '32' (which is ASCII '$'). So the only safe option is to disconnect and reconnect.

What is the video source that is causing this?

[oleaasbo]

I use D-Link Vigilance DCS-4602.

How do you suggest to catch this error? (To do a reconnect)

[oleaasbo]

In regards to the reconnect issue. Data frame errors happen, nothing to do with that. But we need a simple way to reconnect the stream. Is this implementation any good? I have not reviewed all the code in this library so contributors can tell if there is a better way. If anyone could expand this to actually do a reconnect that would be nice! :) Is calling client.connect again without creating a new instance of the object ok?

[RogerHardiman]

Hi That looks like a good implementation. For auto-reconnect I will probably have to hand back to @mbullington but we would need a flag so it could be disabled. I work on some systems where the RTSP address is dynamic so once the connection is dropped the RTSP address is no longer valid.

[oleaasbo]

I did some more testing and managed to pull some data when this error happens.

Bug in RTSP data framing, please file an issue with the author with stacktrace.
ReadyState -> 0
Data[index] -> 43
Data -> {"type":"Buffer","data":[36,0,5,180,128,96,174,244,22,174,42,169,0,253,212,182,60,129,227,0,6,2,55,3,64,40,44,67,241,118,88,190,13,215,140,51,153,136,54,196,150,50,209,38,51,212,69,151,131,189,134,132,160,66,25,40,78,199,22,47,193,252,27,42,135,37,137,41,16,126,175,134,213,179,100,37,80,17,196,235,16,198,19,54,62,143,222,12,235,165,223,177,186,140,26,52,230,208,15,194,254,245,188,24,45,162,242,18,77,175,149,76,114,97,210,148,175,6,0,102,80,0,144,144,177,230,97,38,252,98,91,183,60,159,209,156,30,184,160,202,41,19,83,125,222,89,86,69,171,27,242,117,99,149,237,148,254,254,216,119,220,57,147,146,36,4,197,206,241,24,4,100,72,123,195,141,152,81,59,142,38,16,215,78,209,128,28,30,183,182,88,168,199,99,138,137,100,69,114,17,88,188,59,251,7,142,88,82,149,33,189,8,186,10,94,54,227,195,161,102,175,112,17,62,230,216,13,113,131,207,122,221,249,242,102,197,161,96,83,215,54,41,234,113,221,73,231,133,94,74,219,135,204,89,70,142,13,105,122,128,139,60,19,196,112,218,94,170,103,129,25,30,236,43,181,92,170,61,140,236,177,58,233,129,84,230,119,13,71,230,25,218,235,130,245,186,111,114,181,190,192,30,160,7,66,29,79,114,13,41,237,11,1,54,12,113,222,94,52,205,136,125,123,106,111,111,228,62,233,128,239,30,119,130,11,110,100,170,114,102,132,242,221,16,36,74,104,83,170,60,203,153,70,54,237,18,216,105,212,22,2,42,50,101,30,204,146,63,220,184,148,147,149,170,234,85,176,206,28,250,50,214,33,236,117,79,208,21,12,164,205,201,112,241,54,227,224,35,49,233,239,112,163,162,151,248,56,78,81,0,62,126,174,67,23,212,172,17,136,59,108,29,88,22,72,59,115,202,62,3,190,224,226,43,188,114,221,148,163,166,22,45,173,45,16,19,28,249,158,29,63,109,38,224,29,123,146,241,117,131,129,157,137,13,16,190,214,48,100,124,239,155,31,249,157,184,204,94,192,214,115,126,189,228,92,101,147,165,72,139,224,111,80,31,243,199,217,113,216,11,69,155,40,181,214,107,156,35,79,112,169,223,117,198,95,240,140,106,83,47,203,131,127,240,130,121,3,231,40,60,179,73,150,73,142,2,23,94,42,27,237,201,162,55,164,111,19,74,208,58,140,200,222,114,187,55,161,147,51,156,195,164,148,236,140,198,255,24,26,240,212,220,37,74,216,230,114,217,176,180,86,68,68,64,29,62,103,209,183,243,218,221,23,101,235,124,148,36,11,56,92,192,22,82,50,85,240,67,227,194,248,17,244,237,199,23,96,63,167,38,96,9,6,247,186,58,114,217,7,117,240,47,153,41,38,163,215,70,0,121,53,150,147,244,254,162,52,12,105,253,166,146,217,54,254,243,243,16,246,145,248,82,207,136,207,42,148,221,155,173,25,179,56,20,22,113,170,17,247,217,73,4,231,247,193,7,31,201,62,6,39,195,76,252,163,145,10,73,177,100,116,86,180,177,249,23,85,82,237,167,251,150,226,126,50,109,47,143,161,56,170,100,138,76,104,29,52,201,105,178,76,14,13,53,177,7,50,50,207,21,175,117,159,234,226,56,24,50,69,124,65,223,189,251,102,149,57,252,140,232,196,90,11,91,169,106,77,107,62,252,28,192,107,192,202,99,26,212,80,151,141,86,148,231,240,228,156,45,94,238,105,43,41,97,39,80,143,235,227,224,114,183,99,141,233,47,88,118,108,157,208,52,34,151,141,237,249,90,167,162,104,221,236,41,40,187,229,195,253,6,126,207,96,27,40,224,47,130,68,126,181,106,25,166,149,234,241,144,156,162,154,101,145,235,50,234,38,50,58,204,223,38,172,61,62,249,115,18,127,59,219,1,152,86,87,181,222,215,55,102,32,77,134,40,7,170,67,1,25,131,59,161,135,141,72,242,213,16,129,218,187,4,178,39,26,226,4,143,48,94,189,177,8,28,205,213,10,123,110,192,251,82,128,26,109,68,20,199,96,146,17,227,232,13,53,132,132,112,75,20,213,19,91,133,151,211,18,100,171,185,150,121,192,167,42,13,75,201,32,89,150,44,104,104,226,107,127,8,37,149,219,111,241,106,232,195,195,92,20,181,135,43,177,61,114,121,150,14,225,233,113,198,169,227,174,158,237,211,237,186,8,117,22,175,98,191,78,86,196,230,131,218,19,49,252,206,242,92,15,65,81,59,176,83,71,177,91,174,169,110,49,250,79,9,243,16,241,21,150,30,100,28,152,64,152,104,134,144,58,141,67,159,247,201,189,143,143,187,193,67,5,82,69,14,246,253,212,204,243,67,103,35,66,199,174,31,166,118,44,22,1,182,90,189,109,85,223,211,118,186,241,64,243,34,131,175,187,253,10,214,236,248,75,90,4,65,93,236,21,210,163,215,7,30,61,149,201,247,225,152,57,33,101,135,139,83,254,125,20,197,140,117,168,10,193,82,231,246,97,243,14,153,108,90,21,176,139,18,202,76,50,60,77,8,15,13,173,132,53,47,174,72,70,44,101,210,98,236,151,67,187,225,191,115,111,0,237,185,146,173,2,223,7,164,9,204,37,97,85,178,121,113,147,244,226,166,31,73,16,0,123,233,203,100,172,21,53,175,239,218,177,159,76,248,255,59,168,114,82,78,59,248,66,232,207,59,63,184,124,213,170,90,185,251,0,72,138,53,99,226,91,212,104,123,93,186,168,119,14,37,51,239,91,117,213,12,7,61,10,149,190,72,63,49,127,79,151,229,99,187,56,147,98,70,90,81,187,162,120,131,41,230,62,97,138,132,57,152,93,14,174,51,55,215,222,119,31,122,130,160,250,78,235,17,77,198,59,40,228,245,99,174,58,234,59,236,2,53,44,38,201,212,182,57,11,43,4,11,35,10,137,16,18,71,202,129,216,136,248,180,109,43,34,102,158,60,23,74,158,201,116,83,46,37,241,181,32,145,181,65,212,57,131,112,238,108,225,77,2,58,228,210,80,254,116,64,156,219,223,150,173,252,159,1,114,41,106,203,210,27,54,18,80,119,123,72,215,219,130,168,74,42,207,204,31,90,237,182,104,108,164,49,76,151,180,181,111,89,179,215,55,247,19,34,250,113,167,52,232,214,244,60,46,215,104,130,127,15,83,197,201,8,177,137,239,165,224,199,51,253,202,167,57,205,170,166,137,176,85,168,141,15,118,18,36,0,5,180,128,96,174,245,22,174,42,169,0,253,212,182,60,1,63,50,210,188,225,9,94,43,173,199,94,235,113,4,246,190,174,138,235,175,132,171,66,33,46,186,106,176,44,53,215,129,211,67,222,2,1,236,250,158,119,126,82,26,178,56,158,249,235,245,64,142,245,183,201,121,59,125,234,28,169,226,51,5,232,114,64,216,65,87,220,41,245,44,253,170,212,14,201,123,88,222,166,164,110,88,140,70,69,46,181,126,40,139,49,222,79,139,203,68,146,180,95,219,246,9,4,255,249,95,198,215,1,202,143,148,24,182,107,98,165,136,60,41,107,71,202,218,81,48,28,35,162,146,254,155,97,254,111,187,176,111,240,254,126,23,253,90,137,189,163,163,209,225,49,210,153,239,255,81,220,157,232,1,12,87,200,72,139,108,113,230,44,19,237,50,20,76,149,111,106,184,49,64,38,208,3,149,139,186,150,210,79,146,93,190,102,202,230,142,163,39,181,41,219,180,148,188,68,85,177,84,78,255,184,96,26,142,115,98,232,6,16,119,141,43,29,20,235,101,181,91,191,124,204,95,8,113,46,131,65,44,37,55,115,106,189,198,149,44,247,216,23,159,230,184,245,39,250,98,53,126,208,148,123,77,141,103,43,38,170,78,11,90,162,248,39,203,236,20,133,223,195,6,58,122,227,171,4,206,63,213,9,88,214,240,149,4,9,234,164,207,133,202,182,143,59,99,79,239,154,65,244,12,29,38,124,232,50,183,216,136,147,25,150,73,174,198,246,153,220,66,35,56,64,76,145,231,132,235,37,210,53,178,120,205,33,41,131,90,75,89,156,16,128,120,249,88,109,20,252,172,30,155,82,91,199,17,38,141,13,81,235,81,249,38,73,174,248,53,140,176,43,48,113,164,128,228,165,25,101,238,93,233,253,188,178,15,227,93,87,203,85,194,214,28,83,191,42,141,105,67,239,128,191,163,154,252,242,120,222,77,21,38,116,232,69,191,115,95,154,188,148,240,202,44,39,146,159,104,147,116,217,100,94,127,0,188,235,160,72,233,200,45,27,235,85,127,3,191,124,180,223,9,218,129,32,157,50,73,19,227,80,216,244,200,133,92,248,43,25,129,225,152,78,255,166,62,78,112,5,56,72,140,60,3,135,117,176,13,244,83,88,239,92,24,141,74,238,134,158,15,228,13,60,111,141,23,223,73,225,2,193,222,123,156,157,164,233,223,54,202,34,159,21,1,248,130,163,85,40,77,197,45,109,93,119,192,178,18,91,81,232,142,153,170,204,244,201,127,93,15,61,73,136,212,99,158,87,178,118,164,207,139,178,65,178,68,9,138,176,169,93,149,32,158,77,34,212,63,38,196,231,202,190,12,160,161,102,125,237,158,24,221,214,197,222,158,182,11,237,114,191,1,220,196,67,46,196,7,2,67,177,6,109,227,91,15,234,172,221,249,204,212,68,153,25,177,13,46,1,158,187,202,119,172,124,111,55,203,238,54,0,30,136,218,147,26,218,227,129,208,156,58,51,36,234,223,129,166,205,203,61,51,114,133,98,85,24,3,169,239,141,85,251,138,22,162,52,245,80,135,35,177,13,86,159,0,106,110,96,20,139,173,107,240,80,153,246,249,173,212,88,121,28,103,29,143,253,186,38,173,65,94,114,61,241,86,159,60,140,111,210,64,176,72,237,155,96,170,12,173,68,118,68,199,246,39,12,55,31,78,19,119,224,80,57,20,31,152,97,11,227,251,32,103,229,73,182,244,192,43,56,135,136,74,185,78,68,17,95,49,201,142,117,136,129,203,46,224,33,135,105,128,50,172,155,148,227,243,48,112,48,229,25,59,128,214,142,60,84,93,132,62,68,194,153,26,191,185,22,33,0,81,135,12,118,146,44,233,24,82,243,248,37,114,150,73,198,243,138,220,81,20,102,87,224,4,200,173,246,92,228,96,132,65,164,20,55,49,169,167,179,50,213,146,13,248,210,231,251,54,15,169,158,152,164,180,235,54,91,66,194,55,244,243,51,237,27,4,95,140,223,174,216,155,4,13,31,222,211,34,133,3,229,130,80,250,132,38,131,214,56,81,62,118,86,107,207,218,133,26,96,121,43,152,91,125,10,112,207,114,83,150,151,227,43,200,197,31,209,121,141,2,151,91,224,107,20,133,147,154,195,77,143,139,14,252,142,203,68,143,161,15,216,75,43,27,63,169,105,130,65,242,164,239,55,231,121,216,235,203,46,145,179,251,19,235,69,49,151,44,123,179,12,176,224,143,105,232,54,32,113,36,12,160,167,48,109,4,17,116,236,233,174,203,101,167,250,93,182,160,91,128,190,96,9,214,156,2,100,17,175,124,253,132,8,85,102,82,250,123,169,90,233,56,215,67,21,75,205,206,190,158,133,170,177,52,92,192,46,156,12,70,33,91,115,73,212,122,103,92,124,231,110,167,240,30,203,224,142,136,154,99,41,49,59,40,205,79,177,140,116,115,32,225,112,19,164,61,0,101,100,93,12,132,40,38,45,108,82,85,201,246,9,189,212,248,253,32,200,66,74,79,6,177,207,185,228,150,56,87,92,67,114,119,226,132,31,220,242,203,226,185,235,172,198,73,21,179,208,125,147,186,49,114,134,64,94,79,113,135,80,205,198,203,193,170,162,162,114,161,219,240,82,196,236,63,71,249,92,133,219,50,215,243,64,27,112,21,73,228,2,118,49,195,103,107,131,200,178,247,104,10,93,91,114,175,115,38,69,137,102,88,146,183,233,244,91,36,81,186,254,133,246,126,111,124,239,140,232,141,179,71,141,72,9,20,150,38,123,96,129,211,22,170,132,44,216,79,12,33,186,115,190,40,129,109,43,56,250,111,209,217,180,93,106,217,5,224,202,224,95,140,129,86,60,32,193,17,68,227,206,167,75,73,142,210,106,114,164,215,75,241,30,233,123,29,202,188,80,216,7,246,24,129,49,190,130,4,116,143,46,238,19,163,12,197,253,22,112,52,198,211,204,25,191,0,179,210,7,92,84,69,226,245,35,70,191,167,155,71,216,69,80,71,215,222,129,19,98,168,56,10,107,93,42,130,158,131,98,134,61,61,171,234,233,72,132,21,4,60,110,58,200,217,248,62,97,77,5,28,115,48,236,167,91,139,93,125,128,63,254,1,246,168,201,30,181,254,191,116,173,32,230,202,158,187,92,133,142,148,97,249]}

Here is the console.log that creates the output:

console.log("Bug in RTSP data framing, please file an issue with the author with stacktrace.");
console.log('\x1b[31mReadyState -> \x1b[0m' + this.readState);
console.log('\x1b[31mData[index] -> \x1b[0m' + data[index]);
console.log('\x1b[31mData -> \x1b[0m' + JSON.stringify(data));

EDIT: I extracted the Yellowstone code that interprets the buffer. From what I can see there are 2 packets in the buffer, both with an intended length of 1460 bytes. The last packet however only contains 1428 bytes. I honestly do not know how data[index] ended up as 43 with ReadyState at 0 (Searching). I am unable to duplicate the error using the buffer and Yellowstone logic (i copied the _onData() from RTSPClient.js and inserted the buffer).

However, I removed the "throw error" if none of the if-statements matches (Within _onData()) and changed it to a "break" to throw that buffer away and keep the data flowing. I will test and see if the error at least does not crash Yellowstone with this change. View Changes

[oleaasbo]

I can now confirm that just throwing the bad buffer is the way to go. At least it works. The changes I linked in the post above contains another commit which can be ignored. Whenever a bad buffer appears, do not throw an error. Just break out of the while loop.