MushMal
commented
3 years ago Why are you not using absolute timestamp mode? In relative mode, every session will start from 0 - for example, if the SDK does token rotation or if the connection drops, the new video stream will have 0 as the first frame timestamp. Some applications are OK with it but most cases would use absolute timestamp.
At any rate, taking a quick look at your application code there really isn't anything wrong that I can spot immediately. However, when I looked at the picture of the snippet of logs, I can see the following
- The streaming is 17 seconds behind - likely an issue with networking throughput or the rollback as you have errors
- There were 4 errors in the stream - not sure what they were as there is no log attached
- Your multi-track does not produce frames from all of the tracks or the timelines are different. There is an error 0x52000085 returned from a put frame call.
0x52000085 status code is STATUS_MAX_FRAME_TIMESTAMP_DELTA_BETWEEN_TRACKS_EXCEEDED which is defined in
This is an indication that the frame order coordinator buffer is overflown. This happens when the coordinator accumulates the frames from one track until it can interleave the frames from the other track so the timestamps will be monotonically increasing. That small buffer size is 65 frames by default. What's likely happening is that either your audio or video is not producing frames at a given cadence or your audio timestamps are on a different timelines than the video frames. For example, if the audio is using a clock that starts from 0 whereas the video encoder is using the wall clock or something like that. The GStreamer plugin attempts to compensate for all these issues. You might need to compensate for your timeline drift to ensure audio and video frames are on the same timeline so the frame order coordinator can properly reorder/interlace the audio video frames.
This said, I am not yet sure whether this is the root cause of your decoding issue. There are too many unknowns.
Try to capture longer logs. Analyze your logs and see what's going on. Dump the timestamps and ensure they are on the same timeline. Ensure the encoder is producing both audio and video frames. Note that between the video frames there can not be more than 65 audio frames. Of course, if your case requires, you could change the default 65 frame local frame order coordinator buffer to something larger at the expense of more memory but usually it's an indication of a bad media pipeline.
Please update the info with your findings. Hope the pointers I gave will help you debug your media pipeline in more details.
yibin-lin
disa6302
Hi, I want to push h264 video and aac audio streams to Amazon server, but I still get the error message as the title shows. I have tried that if only the h264 video stream is pushed, it can be displayed successfully, such as the picture below. I don't know where my AAC audio stream causes the server to decode errors. I have also tried to write the AAC of IPC Camera as a file, and it can be played normally on the player. I also tried to change the timestamp of AAC, but no matter which method I tried, I still encountered this error. I am eager for your help, because from the client side, I can't think of what I did wrong. Below are my code and printed error messages, which may take up your precious time to help me analyze where the problem is. thank you very much.
This is a picture of a separate push video stream： `
This is the printing when the video stream and the audio stream are pushed at the same time. There are printings I added and printing generated by calling interface functions：
` The following picture is the media information displayed on the Amazon server when the video stream and audio stream are pushed together：
Below is the code I wrote：
include <com/amazonaws/kinesis/video/cproducer/Include.h>
include "ffkvsc_module.h"
include "ffavs_audio.h"
ifdef __cplusplus
extern "C" {
endif
define DEFAULT_RETENTION_PERIOD 2 * HUNDREDS_OF_NANOS_IN_AN_HOUR
define DEFAULT_BUFFER_DURATION 120 * HUNDREDS_OF_NANOS_IN_A_SECOND
define DEFAULT_CALLBACK_CHAIN_COUNT 5
define DEFAULT_KEY_FRAME_INTERVAL 45
define DEFAULT_FPS_VALUE 25
define DEFAULT_STREAM_DURATION 20 * HUNDREDS_OF_NANOS_IN_A_SECOND
define DEFAULT_STORAGE_SIZE 16 1024 1024
define RECORDED_FRAME_AVG_BITRATE_BIT_PS 3800000
define SAMPLE_AUDIO_FRAME_DURATION (/20/128 * HUNDREDS_OF_NANOS_IN_A_MILLISECOND) //1024/8000
define SAMPLE_VIDEO_FRAME_DURATION (HUNDREDS_OF_NANOS_IN_A_SECOND / DEFAULT_FPS_VALUE)
define AUDIO_TRACK_SAMPLING_RATE /48000/8000
define AUDIO_TRACK_CHANNEL_CONFIG 2
define FILE_LOGGING_BUFFER_SIZE (100 * 1024)
define MAX_NUMBER_OF_LOG_FILES 5
static ffss_thread_t gs_kvsc_stream_task; static ffss_thread_t gs_kvsc_video_task; static ffss_thread_t gs_kvsc_audio_task; static ffstream_t gs_kvsc_video_stream; static ffstream_t gs_kvsc_audio_stream; bool gs_brun = false;
typedef struct { PBYTE buffer; UINT32 size; } FrameData, *PFrameData;
typedef struct { ATOMIC_BOOL firstVideoFramePut; UINT64 streamStopTime; UINT64 streamStartTime; STREAM_HANDLE streamHandle; CLIENT_HANDLE clientHandle; FrameData audioFrames[NUMBER_OF_AAC_FRAME_FILES]; FrameData videoFrames[NUMBER_OF_H264_FRAME_FILES];
} SampleCustomData, *PSampleCustomData;
SampleCustomData g_data;
// Forward declaration of the default thread sleep function VOID defaultThreadSleep(UINT64);
STATUS ffkvsc_read(PFrame pFrame, ffstream_t pHndl) { frame_info_t vinfo = NULL; STATUS retStatus = STATUS_SUCCESS;
CleanUp: return retStatus; }
PVOID PutVideoFrameThread(PVOID args) { ffss_thread_set_name("KVS-Video"); STATUS retStatus = STATUS_SUCCESS; Frame frame; STATUS status; UINT32 frameIndex = 0; int result;
CleanUp: G_ipcData->cloud.login_state = 0; if(retStatus != STATUS_SUCCESS) { kvs_wrn("CleanUp PutVideoFrameThread failed with 0x%08x", retStatus); } return (PVOID) (uintptr_t*) retStatus; }
PVOID PutAudioFrameThread(PVOID args) { ffss_thread_set_name("KVS-Audio"); STATUS retStatus = STATUS_SUCCESS; Frame frame; STATUS status; int result;
CleanUp: if (retStatus != STATUS_SUCCESS) { kvs_wrn("PutAudioFrameThread failed with 0x%08x", retStatus); } return (PVOID) (uintptr_t*) retStatus; }
static int load_kvsc(void) { PDeviceInfo pDeviceInfo = NULL; PStreamInfo pStreamInfo = NULL; PClientCallbacks pClientCallbacks = NULL; PStreamCallbacks pStreamCallbacks = NULL; CLIENT_HANDLE clientHandle = INVALID_CLIENT_HANDLE_VALUE; STREAM_HANDLE streamHandle = INVALID_STREAM_HANDLE_VALUE; STATUS retStatus = STATUS_SUCCESS; PCHAR accessKey = NULL, secretKey = NULL, sessionToken = NULL; PCHAR streamName = NULL, region = NULL, cacertPath = NULL; PTrackInfo pAudioTrack = NULL; BYTE audioCpd[KVS_AAC_CPD_SIZE_BYTE]; UINT64 streamStopTime, streamingDuration = DEFAULT_STREAM_DURATION;
CleanUp: G_ipcData->cloud.login_state = 0; if (STATUS_FAILED(retStatus)) { G_ipcData->cloud.login_state = 2; kvs_err("Failed with status 0x%08x\n", retStatus); } if (LIKELY(pDeviceInfo != NULL)) { freeDeviceInfo(&pDeviceInfo); } if (LIKELY(pStreamInfo != NULL)) { freeStreamInfoProvider(&pStreamInfo); } if (IS_VALID_STREAM_HANDLE(streamHandle)) { freeKinesisVideoStream(&streamHandle); } if (IS_VALID_CLIENT_HANDLE(clientHandle)) { freeKinesisVideoClient(&clientHandle); } if (LIKELY(pClientCallbacks != NULL)) { freeCallbacksProvider(&pClientCallbacks); }
return 0; }