Closed staowyze closed 3 years ago
0x52000084 is defined here: https://github.com/awslabs/amazon-kinesis-video-streams-pic/blob/master/src/client/include/com/amazonaws/kinesis/video/client/Include.h#L181
This status code is returned when you have a multi-track stream, you are using EoFR frame to indicate the termination of the fragment and you are using key-frame flag in the frames. When using multi-track, we can't handle EoFR with ease due to frame reordering. Seems like your application is setting a key-frame flag and the EoFR? Are you running one of the stock samples and modifying it?
Exactly, we set the FRAME_FLAG_KEY_FRAME flag by accident. However, after removing the FRAME_FLAG_KEY_FRAME flag, the video doesn't look right, we send EoFR when a video clip is over, however, the video clip could have multiple Key frames, does this sound right?
There are two separate issues.
1) How do you drive the fragmentation. If you specify keyFrameFragmentation = TRUE in StreamInfo.StreamCaps structure then the fragments will be cut whenever you specify a key frame flag. This is useful with video encoders. Your application could configure the encoder to produce say 4 second GoPs and then specify key frame flag with each of the IDR frames. The audio tracks then will be added without any key frame flags. The frame order coordinator will reorder frames to make those monotonically increasing for the final MKV cluster as this is a requirement. 2) When your clip is done you should submit EoFR to ensure that when new clip is produced, the previous fragment won't be retried. This comes from the fact that when the last fragment was sent, no ACKs were generated as the backend doesn't know whether there are more frames coming. After the timeout, the connection gets closed and the state machine goes to ready state (as there is nothing to be sent). With the new frame, a new streaming session is created, however, as there hasn't been a persisted ACK for the previous fragment, there will be rollback to re-stream the previous fragment. This could cause latency pressure firing which could cause re-setting the connection which, in turn, would end up with invalid MKV on the not-yet used connection that's being closed (it's benign).
I would like to understand whether EoFR in multi-track in your case (with key frame fragmentation and all audio frames without any flags) causes the 0x52000084 status code?
We use video key frames to drive fragment, and audio frames don't have any flags, then we send an single EoFR frame at the end of the video clip, we get 0x52000084 error code right after sending EoFR.
Got it. Let me dive into this scenario ASAP. Will update soon. Can you confirm that you have key frame fragmentation set to TRUE in the StreamInfo?
Yes, confirmed, here is our piece of pseudo code
frame.version = FRAME_CURRENT_VERSION;
if (MEDIA_TYPE_VIDEO)
{
frame.trackId = DEFAULT_VIDEO_TRACK_ID;
if (MEDIA_CODEC_H264_I)
{
frame.flags = FRAME_FLAG_KEY_FRAME;
}
else
{
frame.flags = FRAME_FLAG_NONE;
}
}
else if (MEDIA_TYPE_AUDIO)
{
frame.trackId = DEFAULT_AUDIO_TRACK_ID;
}
Confirming, we have an overzealous check for the key-frame in multi-track case with frame order coordinator. Working on a fix for this particular case. The existing logic works with multi-track EoFR in case when the EoFR is driving the fragmentation itself so the fragments are fragmented by the EoFR itself.
In the existing code, you could set the multi-track, set key-frame fragmentation but mark no fragments as key-frames. Just specify EoFR on each fragment end.
While this usage is good for some real-life use cases, this doesn't work as clean in your case with marking clip boundaries in multi-track. This is what we will try to fix without breaking the syntactic validity of the underlying MKV in case of frame reordering.
Will resolve this as soon as we have the intermittent producer scenario commited
Resolving since intermittent producer is available on master.
Hi experts,
we are seeing 0x52000084 errors with SDK 3.0, any ideas what could be the cause? here is an sample log