search

awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
373 stars 336 forks source link

SDK version[3.0.0] Dropped Frame/Fragment[QUESTION] #642

Closed derek1875 closed 3 years ago

derek1875 commented 3 years ago

Hi, I am testing 3.0.0 SDK and a few video/audio frames at the beginning of a stream may be dropped. could you please check this log?

More information would be provided if you needed. Thanks.

disa6302 commented 3 years ago

@derek1875 ,

What I notice is that your network has a problem:

curlCompleteSync(): curl perform failed for url https://s-4010bf70.kinesisvideo.us-west-2.amazonaws.com/putMedia with result Timeout was reached: Operation too slow. Less than 30 bytes/sec transferred the last 30 seconds

Also, when you say stop sending frames for a while, do you call the stopStreamSync` or the application just does not send frames and sleeps?

Are there any setting deviations from the defaults?

derek1875 commented 3 years ago

Thanks disa6302.

I do not call the stopStreamSync, and my application just stop sending frames and sleep.

Here is my major configuration and pseudo streaming logic:

PStreamInfo pstreamInfo;
CLIENT_HANDLE clientHandle;
PDeviceInfo pDeviceInfo;
STREAM_HANDLE streamHandle;
PClientCallbacks pClientCallbacks;
PCHAR defaultRegion = (PCHAR) "us-west-2";
UINT64 storageSize = 9 * 1024 * 1024;

void kinesis_video_init()
{
    CHK_STATUS(createRealtimeAudioVideoStreamInfoProvider(streamName,
                                                              2 * HUNDREDS_OF_NANOS_IN_AN_HOUR,
                                                              60 * HUNDREDS_OF_NANOS_IN_A_SECOND,
                                                              &pstreamInfo));
    for (i = 0; i < pstreamInfo->streamCaps.trackInfoCount && pstreamInfo->streamCaps.trackInfoList[i].trackType != MKV_TRACK_INFO_TYPE_AUDIO; ++i);
    CHK(i < pstreamInfo->streamCaps.trackInfoCount, STATUS_TRACK_INFO_MISSING);
    pTrackInfoAudio = pstreamInfo->streamCaps.trackInfoList + i;
    STRNCPY(pTrackInfoAudio->codecId, (PCHAR) "A_MS/ACM", MKV_MAX_CODEC_ID_LEN);
    pTrackInfoAudio->codecPrivateData = audioCpd;
    pTrackInfoAudio->codecPrivateDataSize = SIZEOF(audioCpd);

    STRNCPY(pstreamInfo->streamCaps.contentType, (PCHAR) "video/h264,audio/alaw", MAX_CONTENT_TYPE_LEN);
    pstreamInfo->streamCaps.frameRate = 130; 
    pstreamInfo->streamCaps.nalAdaptationFlags = NAL_ADAPTATION_ANNEXB_NALS | NAL_ADAPTATION_ANNEXB_CPD_NALS;
    pstreamInfo->streamCaps.absoluteFragmentTimes = TRUE;

    CHK_STATUS(createDefaultDeviceInfo(&pDeviceInfo));
    CHK_STATUS(setDeviceInfoStorageSize(pDeviceInfo, storageSize));
    pDeviceInfo->clientInfo.loggerLogLevel = LOG_LEVEL_ERROR;//LOG_LEVEL_VERBOSE;
    CHK_STATUS(createDefaultCallbacksProviderWithIotCertificate(endPoint,
                                                                        IOT_CERT_PATH,
                                                                        IOT_PRIVATE_KEY_PATH,
                                                                        IOT_CA_CERT_PATH,
                                                                        roleAlias,
                                                                        thingName,
                                                                        defaultRegion,
                                                                        NULL, NULL, &pClientCallbacks));

    CHK_STATUS(createStreamCallbacks(&pStreamCallbacks));
    pStreamCallbacks->customData = (UINT64)NULL;
    pStreamCallbacks->bufferDurationOverflowPressureFn = NULL;
    pStreamCallbacks->streamLatencyPressureFn = NULL;
    pStreamCallbacks->streamConnectionStaleFn = NULL;
    pStreamCallbacks->droppedFrameReportFn = defaultDroppedFrameReportCallback;
    pStreamCallbacks->droppedFragmentReportFn = defaultDroppedFragmentReportCallback;
    pStreamCallbacks->streamErrorReportFn = defaultStreamErrorReportCallback;
    pStreamCallbacks->fragmentAckReceivedFn = defaultFragmentAckReceivedCallback;
    pStreamCallbacks->streamReadyFn = NULL;
    pStreamCallbacks->streamClosedFn = NULL;
    pStreamCallbacks->streamShutdownFn = NULL;
    pStreamCallbacks->freeStreamCallbacksFn = NULL;

    CHK_STATUS(addStreamCallbacks(pClientCallbacks, pStreamCallbacks));

    PlatformCallbacks platformCallbacks = {0};
    platformCallbacks.logPrintFn = myLogging;
    CHK_STATUS(setPlatformCallbacks(pClientCallbacks, &platformCallbacks));
    CHK_STATUS(createKinesisVideoClient(pDeviceInfo, pClientCallbacks, &clientHandle));
    CHK_STATUS(freeDeviceInfo(&pDeviceInfo));

    createKinesisVideoStream(clientHandle, pstreamInfo, &streamHandle);
}

int main()
{
    kinesis_video_init();

    while(1) {
      //video fps is 15, so actually we are pushing a 12s video. 
      for(int i=0; i < 180; i++) {
        get_frame(frame);
        putKinesisVideoFrame(streamHandle, frame);
      }
      sleep(30);
    }
}

On my device it works well at most of the time but the question is a few video/audio frames at the beginning of a video may be dropped. and what I confirmed is that the network is not in a trouble.

So how should I do to avoid [KVS] contentViewTrimTailItems(): ContentView is not big enough to contain a single fragment. Thanks a lot.

MushMal commented 3 years ago

RE: [KVS] contentViewTrimTailItems(): ContentView is not big enough to contain a single fragment

This is a warning with the default drop frame policy. What it means is that if the content view needs to drop a frame for any reason (due to temporal pressures like in your case where you have an intermittent producer case) it will not only drop a single frame but will continue dropping all of the frames of the same fragment. This makes sense for video as dropping for example an I-frame only will lead to semantically invalid fragment so the attempt is made to remove all of the P-frames after the frame that's being dropped. This is a warning.

What happens in your scenario is that you have an "unenlightened" intermittent producer. As you simply pause and don't put frames, the session times out. Next time you produce a frame, the state machine will attempt to re-stream the last fragment as it hasn't been ACK-ed and will end up in a situation of max latency breach which is a latency pressure. The frame will be dropped and this warning will be caused.

This commit should fix the issue with the automatic intermittent producer scenario: https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/pull/640

Please update to this and give it a try.

Capture verbose logs and attach as a file to the issue if you happen to have any problems. Please provide detailed description of the issue with the detailed description of your scenario and settings and whether you've modified any code.

derek1875 commented 3 years ago

@MushMal Thanks for your answer. I'm doing test on the commit you recommended and will attached logs once the problem happened.

disa6302 commented 3 years ago

@derek1875 ,

I am closing this ticket. If you face any issues with intermittent streaming, feel free to open a new issue with the question.