[BUG] 0x52000062 during streaming

[imizus]

Describe the bug Error 0x52000062 during the continuous streaming

To Reproduce Running the application, that is streaming the video with 30 fps to KVS and streaming. At some point during a continuous uploading of the frames, 0x52000062 (MKV Header Error) error occurs and after that if the application tries to continue streaming, it receives multiple 0x5200004a (End Of Stream) errors and drops frames. Then the connection has to be re-established.

SDK version number 3.0.0

Platform (please complete the following information):

• OS: Linux
• Processor: Intel Xeon Platinum 8176
• Version: Ubuntu 16.04.4

Additional context Problem occurs sporadically once in some streaming hours

Log part attached. There were no errors or dropped frames in the log before the beginning of the chunk. 62_and_4a.txt

[MushMal]

@imizus 0x5200004a (https://github.com/awslabs/amazon-kinesis-video-streams-pic/blob/master/src/client/include/com/amazonaws/kinesis/video/client/Include.h#L124) is the End of Stream indicator - it's not an error but a way the PIC communicates to upstream that the particular session/stream is closed. In 3.0.0 this was being printed out as a warning/error in the logs but this was fixed a few commits later - ignore this.

0x52000062 (https://github.com/awslabs/amazon-kinesis-video-streams-pic/blob/master/src/client/include/com/amazonaws/kinesis/video/client/Include.h#L147) is returned from the backend indicating that it encountered an error parsing the stream.

Looking at the logs, I see a few instances of the invalid MKV error but the logs are filled with your application output and the KVS output is not the debug output which makes it hard to understand what's going on.

Prior to the invalid MKV errors, there is a series of buffer pressure events and once this error occurs the higher-level logic attempts to recover and re-stream. The streaming then gets terminated with a large number of frames being dropped from the buffer.

Can you include a little more information about the frequency of this issue occurring? Is this happening with every stream for you? Can we get debug logs to see what could cause this issue and the current buffer information that's being output periodically? Also, could you please include the stream definition and the device info to see if there are any issues with the non-default settings you might have set?

[imizus]

output is not the debug output which makes it hard to understand what's going on.

How to make it a debug output? Because just building KVS libraries in the "Debug" mode seems insufficient.

Could you please give some instruction?

About the frequency - it occurs once in, say, 50 runs.

Stream definition:

    config.m_streamMaxLatency = seconds(15);
    config.m_fps = 30;
    config.m_bitrate = 120000000;

    auto stream_definition = make_unique<StreamDefinition>(config.m_streamName.c_str(),
        config.m_streamRetention,
        nullptr,//&tags,
        "",
        STREAMING_TYPE_REALTIME,
        "video/h264",
        config.m_streamMaxLatency,
        seconds(1), // fragment duration
        milliseconds(1),
        true, // key frame fragmentation
        true, // frame_timecodes
        true, // absolute timecodes
        true, // application fragment acks required
        true, // recover after an error
        true, // recalculate metrics at runtime
        NAL_ADAPTATION_FLAG_NONE,
        config.m_fps,
        config.m_bitrate);

[lherman-cs]

@imizus you can bump up the log level by setting AWS_KVS_LOG_LEVEL environment variable to 2

[MushMal]

@imizus the settings look ok to me. Two outstanding questions:

1) The default buffer duration is 120 seconds https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/src/StreamDefinition.h#L66 - is this what you guys need as your max_latency is set to 15 seconds. 2) Per above, your max_latency is set to 15 seconds so any time you acquire 15 seconds worth of frames that have not yet been sent, the default continuous retry logic in C producer layer will attempt to re-set the connection. What's your actual requirement? In itself 15 seconds is OK but it's too aggressive for a lot of the scenarios where a network hiccup can happen.

I am looking forward for the debug logs to see what could cause the INVALID_MKV error ACK

[MushMal]

Resolving due to no response. Please feel free to reopen or cut a new issue