bytedeco / javacv

Java interface to OpenCV, FFmpeg, and more
Other
7.59k stars 1.59k forks source link

Javacv how to parse PS flow? #996

Open gggitpl opened 6 years ago

gggitpl commented 6 years ago

Javacv how to parse PS flow? Can give a sample code?

saudet commented 6 years ago

What is "PS flow"?

gggitpl commented 6 years ago

RTP payload PS flow (RTP荷载PS流)

The PS encapsulation for H264 is as follows: Each IDR NALU will generally contain NALUs such as SPS, PPS, etc. Therefore, the NALUs of SPS, PPS, and IDR are encapsulated into one PS packet, including the ps header, and then the PS system header is added. PS system map PES header+h264 raw data. So the order of an IDR NALU PS packet from outside to inside is: PSheader | PS system header | PS system Map | PES header | For PS packets of other non-key frames, it is much simpler. Directly add the PS header and PES header. The order is: PS header | PES header | h264raw data. The above is the case with video only. If you want to pack Audio Audio into the PS package, you can. When there is audio data, it is OK to put the data plus the PES header on the video PES. The sequence is as follows: PS packet = PS header | PES (video) | PES (audio), and then send it in RTP encapsulation.

针对H264 做如下PS 封装:每个IDR NALU 前一般都会包含SPS、PPS 等NALU,因此将SPS、PPS、IDR 的NALU 封装为一个PS 包,包括ps 头,然后加上PS system header,PS system map,PES header+h264 raw data。所以一个IDR NALU PS 包由外到内顺序是:PSheader| PS system header | PS system Map | PES header | h264 raw data。对于其它非关键帧的PS 包,就简单多了,直接加上PS头和PES 头就可以了。顺序为:PS header | PES header | h264raw data。以上是对只有视频video 的情况,如果要把音频Audio也打包进PS 封装,也可以。当有音频数据时,将数据加上PES header 放到视频PES 后就可以了。顺序如下:PS 包=PS头|PES(video)|PES(audio),再用RTP 封装发送就可以了。

saudet commented 6 years ago

I see, thanks! We'll need someone to include these kinds of features. Would you like to give it a try?

gggitpl commented 6 years ago

I am very willing to try, but I do not know how to start this aspect I also just contact, see FFmpeg can solve the PS stream so ask whether there is achieved

gggitpl commented 6 years ago

I am Chinese. I cannot speak English. I am sorry to use Google.

saudet commented 6 years ago

It looks like a callback was added for this recently: http://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219581.html But it doesn't seem that FFmpeg offers very good support unfortunately...