search

bluenviron / mediamtx

Ready-to-use SRT / WebRTC / RTSP / RTMP / LL-HLS media server and media proxy that allows to read, publish, proxy, record and playback video and audio streams.
MIT License
11.91k stars 1.5k forks source link

Err: PTS is missing while using source udp stream #2103

Open Creepios opened 1 year ago

Creepios commented 1 year ago

Which version are you using?

v0.23.8

Which operating system are you using?

Describe the issue

When connecting a udp source stream using a gopro webcam udp stream it continuously says that 2 tracks a ready and after that ERR: PTS is missing

Describe how to replicate the issue

  1. Enable webcam stream on gopro via api.
  2. Configure mediamtx to read udp stream from own ip
  3. start mediamtx

Did you attach the server logs?

yes

log.txt

Did you attach a network dump?

no

Creepios commented 1 year ago

Probably helpful information: (GoPro Webcam UDP Stream) image

config: image

aler9 commented 1 year ago

Hello, we need a sample of the stream, that can be generated in this way:

If the bug arises when using MediaMTX with an external hardware or software, the most helpful content you can provide is a dump of the data exchanged between the server and the target (network dump), that can be generated in this way: 1) Download wireshark (https://www.wireshark.org/) 2) Start capturing on the interface used for exchanging UD 3) Start the server and replicate the issue 4) Stop capturing, save the result in .pcap format 5) Attach

pvalsecc commented 1 year ago

The issue comes from there: https://github.com/bluenviron/mediacommon/blob/10b74a4f6eda507d2d693e69c063bd1ee595821d/pkg/formats/mpegts/reader.go#L176

It should be OK to have no PTS for some PES packets. In the case of my stream:

I have to put a dummy PTS on both to have mediamtx accept my stream.

In general, when there is a decoding error, the stream should not be closed. We don't want to have deconnections each time an UDP packet is lost by the network.

aler9 commented 1 year ago

Hello @pvalsecc,

It should be OK to have no PTS for some PES packets

the reason why PTS is required is that this server is able to convert streams from a protocol to another, PTS is required in order to do so. If this project was a simple UDP listener, there would be no issues, and the same is valid for a lot of protocol-specific features that are limited in order to guarantee interoperability.

A fast workaround consists in setting missing PTSs equal to the current system time, but we can't make this assumption for all streams, for a lot of reasons:

Therefore, in the future we may add an algorithm able to check the validity of the incoming PTS and eventually replace it with the system timestamp when missing or invalid.

The packets that contains the SPS and the PPS that doesn't have any PTS.

This is one special case we can handle in the future.

I have a PRIVATE_STREAM_1 (some KLV metadata in another PID) that is not having any PPS as well.

That's a case in which we can assume that PTS is equal to the reception timestamp, but be careful that such tracks can't be correctly synchronized with other media tracks. If a video frame has PTS = 30 and a metadata frame has a missing PTS, even if they are being received at the same time, there's no guarantee that the metadata frame corresponds the video one.

In general, when there is a decoding error, the stream should not be closed.

Exactly, that's why decoding errors are not treated as errors but redirected to a callback that prints errors on screen without disconnecting the stream:

https://github.com/bluenviron/mediacommon/blob/10b74a4f6eda507d2d693e69c063bd1ee595821d/pkg/formats/mpegts/reader.go#L100-L102

Lack of PTS is currently treated as an error, but will be soon treated as a decoding error.

Creepios commented 1 year ago

Hello, we need a sample of the stream, that can be generated in this way:

If the bug arises when using MediaMTX with an external hardware or software, the most helpful content you can provide is a dump of the data exchanged between the server and the target (network dump), that can be generated in this way:

  1. Download wireshark (https://www.wireshark.org/)
  2. Start capturing on the interface used for exchanging UD
  3. Start the server and replicate the issue
  4. Stop capturing, save the result in .pcap format
  5. Attach

Is this still necessary after the previous replies? It sounds like the problem has now been identified. Its probably exactly my problem as it seems like that a GoPro camera does not offer any PTS. Funny about that is that VLC Media Player can convert these streams without problems.

pvalsecc commented 1 year ago

Exactly, that's why decoding errors are not treated as errors but redirected to a callback that prints errors on screen without disconnecting the stream:

I'm not so sure about that:

2023/08/03 09:32:30 INF [path patrick] [udp source] ready: 1 track (H264)
2023/08/03 09:33:56 INF [RTSP] [session 8023b20b] is reading from path 'patrick', with UDP, 1 track (H264)
2023/08/03 09:34:07 INF [RTSP] [session ce40d8e2] is reading from path 'patrick', with TCP, 1 track (H264)
2023/08/03 09:34:28 INF [path patrick] [udp source] ERR: astits: building new data failed: astits: parsing PES data failed: astits: fetching next bytes failed: astikit: slice length is 24472, offset 39121 is invalid
2023/08/03 09:34:33 INF [path patrick] [udp source] ready: 1 track (H264)
2023/08/03 09:37:18 INF [path patrick] [udp source] ERR: astits: building new data failed: astits: parsing PES data failed: astits: fetching next bytes failed: astikit: slice length is 14168, offset 34750 is invalid
2023/08/03 09:37:23 INF [path patrick] [udp source] ready: 1 track (H264)

When the PES parsing failed, vlc got disconnected from the RTSP stream and mediamtx started over for my UDP source.

aler9 commented 1 year ago

Funny about that is that VLC Media Player can convert these streams without problems.

VLC is hardened against badly formatted streams and its only role is to show video on screen, not to preserve incoming metadata. Personally i use this server to route video to AI-based processes and UIs, and timing informations must be preserved in both.

Creepios commented 1 year ago

When the PES parsing failed, vlc got disconnected from the RTSP stream and mediamtx started over for my UDP source.

I'm also not sure about the fact that the stream doesn't get closed or disconnected. vlc closed the stream for me as well. Also, I have two mediamtx servers running, where one picks up the source from the other, but the connection still drops completely.

Creepios commented 1 year ago

Funny about that is that VLC Media Player can convert these streams without problems.

VLC is hardened against badly formatted streams and its only role is to show video on screen, not to preserve incoming metadata. Personally i use this server to route video to AI-based processes and UIs, and timing informations must be preserved in both.

Ah thanks for the info, I actually didn't know that vlc isnt preserving metadata. Thats kinda sad, especially for professional use.

pvalsecc commented 1 year ago

I have a PRIVATE_STREAM_1 (some KLV metadata in another PID) that is not having any PPS as well.

That's a case in which we can assume that PTS is equal to the reception timestamp, but be careful that such tracks can't be correctly synchronized with other media tracks. If a video frame has PTS = 30 and a metadata frame has a missing PTS, even if they are being received at the same time, there's no guarantee that the metadata frame corresponds the video one.

The spec (MISB ST 1402.2) I'm following to generate those streams allows two modes for sending metadata:

image

In this mode, the PES packet should not contain any PTS:

image

Creepios commented 5 months ago

Hey, I would like to kindly remind of this issue. Are there already possible approaches or will it be not fixable for unknown time?