Error in ingesting RTMP stream from GPS camera

[studentnumber9]

We have camera-based vehicle trackers that are able to send RTMP streams to a server. The manufacturers provide a media server based on the ZLMediaKit SDK, and it works as expected in receiving the camera stream and allowing it to be viewed by subscribers as rtmp or flv.

We want to implement our own media server based on the LiveStreamingServerNet library. Following the various samples, I have a server running, and the examples of streaming an mp4 file to the server using ffmpeg work like a charm, but when I have the dashcam device send its RTMP stream to the server, I get the error below.

Maybe I'm just missing some required setup, or if you could suggest how I should go about examining what is coming in from the dashcam devices, and what I could do to be able to ingest the incoming RTMP stream. If I have to build a special version of the LiveStreamingServerNet projects to add more logging, I will attempt it, but I will be deeply greatful for any help or steps to try.

info: LiveStreamingServerNet.Networking.Server.Internal.ClientSessionManager[67902755] Connected (ClientId=7) fail: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpChunkEventHandler[698681447] Failed to handle RTMP message (ClientId=7) System.InvalidOperationException: No handler found for message type 0 at LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher`1.DispatchAsync(IRtmpChunkStreamContext chunkStreamContext, TContext context, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpChunkEventHandler.DispatchRtmpMessageAsync(IRtmpChunkStreamContext chunkStreamContext, IRtmpClientSessionContext clientContext, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpChunkEventHandler.HandleRtmpMessageAsync(RtmpChunkEvent event, RtmpChunkMessageAggregationResult aggregationResult, CancellationToken cancellationToken) info: LiveStreamingServerNet.Networking.Server.Internal.ClientSessionManager[208815263] Disconnected (ClientId=7)

[josephnhtam]

@studentnumber9

Zero should not be a standard message type id, which causes this error. I can release a patch to skip any message with unknown type id to enhance the compatibility.

[studentnumber9]

That would be awesome if that resolves the problem...I was afraid the stream had other info in the front like sim id that needed to be stripped or skipped to get to the real rtmp content. I'll test as soon as the patch is available.

[josephnhtam]

@studentnumber9

Please check v0.19.2

[studentnumber9]

Now I don't get the error on message type id of 0, but then the RtmpPublishCommandHandler never fires to publish the stream, I don't think the incoming stream is getting consumed meaningfully...I'll have to dig into exactly what is being sent from the camera device in its supposed rtmp stream.

[josephnhtam]

Can you try turning on the verbose logging to check and share the logs?

using var liveStreamingServer = LiveStreamingServerBuilder.Create()
    .ConfigureLogging(options => options.AddConsole().SetMinimumLevel(LogLevel.Trace))
    .Build();

await liveStreamingServer.RunAsync(new ServerEndPoint(new IPEndPoint(IPAddress.Any, 1935), false));

[studentnumber9]

Here are the extra logs, it seems every chunk is encountering the message type id of 0. It goes on for a while before the divide by zero error.

info: LiveStreamingServerNet.Networking.Server.Internal.ClientSessionManager[67902755] Connected (ClientId=1) dbug: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpHandshakeC0EventHandler[1373722099] Handshake C0 Handled (ClientId=1) dbug: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpHandshakeC1EventHandler[1513154730] Handshake type (ClientId=1, HandshakeType=SimpleHandshake) dbug: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpHandshakeC1EventHandler[1139968112] Handshake C1 Handled (ClientId=1) dbug: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpHandshakeC2EventHandler[1331533433] Handshake C2 Handled (ClientId=1) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=22, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=53, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=56, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=44, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=46, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=19, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=9, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=18, MessageTypeId=0)

....[KEEPS GOING LIKE THIS FOR ANOTHER 350 or so chunks, before error below with divide by zero].....

trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=37104, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=2850, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=235, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=49974, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=57656, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=2431, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=9545, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=32763, MessageTypeId=0) trce: LiveStreamingServerNet.Rtmp.Internal.RtmpEventHandlers.Dispatcher.RtmpMessageDispatcher[525655837] Message handler not found (ChunkStreamId=92, MessageTypeId=0) dbug: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.ProtocolControls.RtmpSetChunkSizeHandler[1761754785] SetChunkSize (ClientId=1, InChunkSize=0) fail: LiveStreamingServerNet.Rtmp.Server.Internal.RtmpClientSessionHandler[1306188046] An error occurred in the client loop (ClientId=1) System.DivideByZeroException: Attempted to divide by zero. at LiveStreamingServerNet.Rtmp.Internal.Services.RtmpChunkMessageAggregatorService.HandleChunkMessagePayloadAsync(IRtmpChunkStreamContext chunkStreamContext, INetworkStreamReader networkStream, UInt32 maxInChunkSize, Int32 headerSize, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Internal.Services.RtmpChunkMessageAggregatorService.AggregateChunkMessagesAsync(INetworkStreamReader networkStream, IRtmpChunkStreamContextProvider contextProvider, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Server.Internal.RtmpEventHandlers.RtmpChunkEventHandler.Handle(RtmpChunkEvent event, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Server.Internal.RtmpClientSessionHandler.HandleChunkAsync(IRtmpClientSessionContext clientContext, INetworkStreamReader networkStream, CancellationToken cancellationToken) at LiveStreamingServerNet.Rtmp.Server.Internal.RtmpClientSessionHandler.HandleSessionLoopAsync(INetworkStreamReader networkStream, CancellationToken cancellationToken) info: LiveStreamingServerNet.Networking.Server.Internal.ClientSessionManager[208815263] Disconnected (ClientId=1)

[josephnhtam]

There seems to be a mismatch in protocol.

Do you know more details about the media server based on ZLMediaKit SDK? Could you also try other RTMP servers like NodeMediaServer or SRS to confirm if the camera is using RTMP protocol? Or could you use Wireshark to capture the network packets?

[josephnhtam]

As a reference, Wireshark can capture the RTMP stream published by OBS Studio like this

[studentnumber9]

The manufacturer is from another country and hard to communicate with, but they say, "the protocol of the stream is rtp", which if true means the task is to consume the RTP packets and convert to RTMP. I am just starting to look into how this could be done, perhaps with ffmpeg, but there is no nice rtp:// URL to the stream, the camera is sent a command and it starts sending packets to an IP/port, which their ZLMediaKit-based media server is able to ingest and republish as rtmp and flv streams.

You have deep expertise in all this, if you could suggest any approaches to try, I will learn and run with it.

In the meantime, I am able to use your RtmpClient to read the rtmp stream from their media server and transmit it to our LiveStreamingServerNet-based server, and it works well. If I cannot pry open the datagrams coming directly from the camera, this relay system would work, but we would like to avoid having to maintain two media servers, and would like more control over the whole streaming process.

I will set up Wireshark to examine the data coming directly from the camera, but again there is no URL to connect to, so will need to figure out how to direct the traffic from camera to Wireshark.

[josephnhtam]

Most IP camera supports RTSP(RTP/RTCP). However, live-streaming-server-net doesn't support RTSP directly at the moment. If the camera is really using RTSP, you may try using ffmpeg with 'ffmpeg -i rtsp://source_ip:554/stream -c copy -f flv rtmp://destination_ip:1935/live/stream'

[studentnumber9]

Well, having the camera send its stream to source_ip:port and then listening for an rtsp or rtp stream from the camera didn't register anything, but I was able to use 'ffmpeg -i tcp://source_ip:port?listen=1 -c:v libx264 -c:a aac -f flv rtmp://destination_ip:1935/live/stream' to receive the camera stream and feed it to our RTMP server. While it technically worked, there are tons of errors in ffmpeg with decoding and the resulting video is very low quality, with lots of packets lost.

It seems I am close, yet so far. I tried the -re and other options, maybe it's a buffering or decoding issue, or maybe the whole process needs to happen in two steps: feed the tcp stream to rtp and then to rtmp. But it's mostly blind flying till I do more research on ffmpeg.

Your excellent library does offer hope for the immediate future in allowing us to consume the internally available ZLMediaKit-based server's rtmp stream and publish it to our external media server that can secure the stream for customers.

[studentnumber9]

@josephnhtam I will mark the issue resolved since there is no problem with LiveStreamingServerNet. Its use as an internal relay system and then an external media server for us works great. If I figure out the ffmpeg commands for successful RTP to RTMP conversion, I will post them for any future travelers. Thank you for your time and suggestions.