search

eerimoq / moblin

Moblin, a free iOS app for IRL streaming.
MIT License
182 stars 20 forks source link

Processing timestamps from DJI correctly #57

Closed ErAzOr2k closed 4 months ago

ErAzOr2k commented 4 months ago

With the adjustment, the timestamps transmitted by DJI are processed correctly. A manual specification of the FPS is therefore obsolete.

The difference from the previous calculation of the timestamps is that video and audio now share a common basis. Previously, both were calculated separately.

The next step would be to find out how we can identify the required calculation at runtime, or make some kind of universal solution from it.

Therefore, it can initially only be activated via the debug settings. When the debug setting is enabled, the previous calculation and specification of the FPS are irgnored.

eerimoq commented 4 months ago

Is there a specification that says timestamps works like this? How did you figure it out?

ErAzOr2k commented 4 months ago

If there is a specification, I would like to have it as well.

In the end, I figured it out through extensive research and experimentation. I examined each packet in Wireshark. A "calculated" timestamp is provided here. It is not possible to achieve the same result through the conventional method (when audio and video are treated separately). Instead, I get the same result when using the same basis as in the current implementation. I compared this with the verbose output in ffmpeg, and it matches.

It is also interesting that DJI initially only outputs video in the first buffers. Here, the expected PTS of 33 is provided. Once audio is added, the deltas of the video buffers become smaller, because audio PTS needs to be added.

eerimoq commented 4 months ago

Would be awesome if you can share the files and commands you used during your research. I want to verify it myself as well.

ErAzOr2k commented 4 months ago

is a bit complex. First you have to install nginx with the RTMP module on your Mac/PC. then both following commands must be executed in parallel:

ffmpeg -i rtmp://127.0.0.1/live/stream -c:v copy -c:a copy -f flv rtmp://moblinip:1935/live/moblin ffmpeg -i rtmp://127.0.0.1/live/stream -vf "showinfo" -f null -

compare number/pts from ffmpeg output und xcode debug output.

I'll send you a wireshark capture in discord.