Closed sheng930920 closed 7 months ago
pedroSG94
commented
11 months ago Hello,
I detected 2 fails that could be the reason because youur server close connection. If you are using H265 you must set the video codec to the RtmpClient like this:
mRtmpClient.setVideoCodec(VideoCodec.H265)Also here:
mRtmpClient.connect(rtmpUrl, true)Replace to:
mRtmpClient.connect(rtmpUrl)You have other error here:
if (videoFrame.isIFrame && !mRtmpClient.isStreaming) {
mVideoInfo.flags = MediaCodec.BUFFER_FLAG_KEY_FRAMEThe flag (only the flag) must be changed in each keyframe not only on first keyframe. This is not totally required but I recommend you do it
sheng930920
commented
11 months ago Thanks for the reply and suggestions. Now I can push the H265 video stream to the SRS streaming server normally. However, I found that it cannot be played. I would like to ask if there is something wrong. Or is the data of sps and pps abnormal?
private fun decodeSpsPpsFromByteArray(videoByteArray: ByteArray): Triple<ByteArray, ByteArray, ByteArray> {
var spsIndex = -1
var ppsIndex = -1
var vpsIndex = -1
var ppsEndIndex = -1
var index = 0
while (index + 3 < videoByteArray.size) {
if (videoByteArray[index].toInt() == 0x00 &&
videoByteArray[index + 1].toInt() == 0x00 &&
videoByteArray[index + 2].toInt() == 0x00 &&
videoByteArray[index + 3].toInt() == 0x01
) {
val nalType = (videoByteArray[index + 4].toInt() and 0x7E) ushr 1
println("nalType: $nalType")
when(nalType) {
// VPS
32 -> vpsIndex = index
// SPS
33 -> spsIndex = index
// PPS
34 -> ppsIndex = index
// PPS
20 -> {
ppsEndIndex = index
break
}
}
}
index++
}
println("vpsIndex: $vpsIndex spsIndex:$spsIndex ppsIndex:$ppsIndex")
val vpsLength = spsIndex - vpsIndex
val spsLength = ppsIndex - spsIndex
val ppsLength = ppsEndIndex - ppsIndex
val vps = ByteArray(vpsLength).apply {
System.arraycopy(videoByteArray, vpsIndex, this, 0, vpsLength)
}
val sps = ByteArray(spsLength).apply {
System.arraycopy(videoByteArray, spsIndex, this, 0, spsLength)
}
val pps = ByteArray(ppsLength).apply {
System.arraycopy(videoByteArray, ppsIndex, this, 0, ppsLength)
}
println("VPS: ${vps.joinToString(", ")}")
println("SPS: ${sps.joinToString(", ")}")
println("PPS: ${pps.joinToString(", ")}")
return Triple(sps, pps, vps)
}The following is the sps and pps data I printed on the logcat console
VPS: 0, 0, 0, 1, 64, 1, 12, 1, -1, -1, 1, 64, 0, 0, 3, 0, -128, 0, 0, 3, 0, 0, 3, 0, 120, -84, 12, 0, 0, 15, -96, 0, 1, -44, -62, 0, -6, 40
SPS: 0, 0, 0, 1, 66, 1, 1, 1, 64, 0, 0, 3, 0, -128, 0, 0, 3, 0, 0, 3, 0, 120, -96, 2, -48, -128, 16, -25, -6, 46, -20, -111, 119, -94, 93, 88, 16, 0, 0, 62, -128, 0, 7, 83, 12, 64
PPS: 0, 0, 0, 1, 68, 1, -63, -83, -16, 19, 100The following text content is the data of I frame. The data of sps, pps and vps are all obtained here. IFrame.txt
pedroSG94
commented
11 months ago Hello,
Your vps, sps and pps method look fine. You can try this:
sheng930920
commented
11 months ago When I try to push the video stream to the MediaMTX streaming server, the push stream is normal, but it still cannot be played using ffplay and VLC players. I see the console output exception message: the stream doesn't contain any supported codec, which are currently H264, MPEG-4 Audio, MPEG-1/2 Audio
pedroSG94
commented
11 months ago Please, try to open url using RTSP or HLS instead of RTMP in player side. It is because RTMP H265 is not supported in player side. Like this:
ffplay rtsp://ip:port/appname/streamname
The player uses RTSP or HLS to play the H265 video stream, but the test results are still not good.
The following is tested using the ffplay command and VLC player
Below is Google Chrome to test hls link playback, but the result is that it cannot be played.
sheng930920
commented
11 months ago Using ffplay settings to force the video to use h265 decoding, the result is the same error
pedroSG94
commented
11 months ago Hello,
Try with ffplay with this command:
ffplay -rtsp_transport tcp rtsp://192.168.0.99:8554/live/3455Also, make sure that you set video resolution to rtmpClient correctly like this:
rtmpClient.setVideoResolution(width, height);The H265 video resolution is correctly set in the code. I use the ffplay command to play the rtsp link and I don’t see any error log output, but I still can’t see the real video screen.
ffplay -rtsp_transport tcp rtsp://192.168.0.99:8554/live/3455override fun onReceive(videoFrame: IVideoFrame) {
mVideoInfo.apply {
size = videoFrame.data.size
offset = 0
presentationTimeUs = System.nanoTime() / 1000
flags = if (videoFrame.isIFrame) MediaCodec.BUFFER_FLAG_KEY_FRAME else 0
}
mRtmpClient.setVideoResolution(videoFrame.width, videoFrame.height)
if (videoFrame.isIFrame && !mRtmpClient.isStreaming) {
decodeSpsPpsFromByteArray(videoFrame.data)?.let { result ->
mRtmpClient.setVideoInfo(ByteBuffer.wrap(result.first), ByteBuffer.wrap(result.second), ByteBuffer.wrap(result.third))
mRtmpClient.connect(rtmpUrl, true)
}
}
if (mRtmpClient.isStreaming) {
mRtmpClient.sendVideo(ByteBuffer.wrap(videoFrame.data), mVideoInfo)
}
}This is weird. Now, I can't see any error
Do you have a way to test using H264 instead H265 to discard that we are doing something bad no related with the codec?
According with others posts h264 should work with DJI
sheng930920
commented
11 months ago When I use an old DJI drone to obtain H264 video stream data for live streaming, everything is normal and the video can be played back. Currently, DJI V5 MSDK and new models of drones can only obtain H265 video data.
pedroSG94
commented
11 months ago I see,
Can you check if you get keyframes continuously and get nalu type of that frames? Maybe the player can't start stream because something is wrong with keyframes (a player normally start to reproduce after receive a keyframe because this contains video info). You can check how to get nalu type here: https://github.com/pedroSG94/RootEncoder/blob/master/rtmp/src/main/java/com/pedro/rtmp/flv/video/H265Packet.kt#L110 Basically apply this line to the first byte after the byte prefix "00 00 00 01" and show the result (do it with all buffers)
Also, I have a last bullet but this means that we need write a code that could be difficult: Do you have a method to decode frames rendering a surface? We can try decode frames and reencode it to get buffers using VideoEncoder class.
pedroSG94
commented
7 months ago Closing as inactive.
When using RtmpClient to push DJI drone H265 video stream data, RtmpSender error java.net.SocketException: Broken pipe, actually my network is available normally
The version information is as follows
implementation 'com.github.pedroSG94.RootEncoder:library:2.3.1'
SRS Server
Below is the code I tested