Player stuck in wait_for_frame() in playing an RTMP link

As mentioned in issue Error with av_gettime() - fix video freezing #81, there is another timing problem in playing an RTMP link (mainly from another country with a big time difference). In this case, the player cannot start to play (turning on logs in player.c shows that the program is stuck in the wait_for_frame() function).

I will explain the cause and my fix below for your comments/suggestions.

The reason is this player->start_time and player->pause_time have been initialized to 0. When we resume the play, we use av_gettime() to get the local time and compute the time difference between av_gettime() and player->start_time and use that as our local video playing time. In the meantime, in decode_video(), we estimate time from the incoming packet. The time difference between the estimated time and 0 (this 0 is implicit) is used as the streaming playing time. The potential problem is when playing an RTMP link from another country with big time difference from the local time, the estimated time is too large (could be negative too even though it has 64 bits to represent a time in us) as compared with av_gettime() - player->start_time.

Fix in player.c (this seems working fine but it is possible I misunderstood something or someone can come up with better ones)

Introduce a time_offset variable and a flag to indicate if the time_offset is acquired.
Initialize time_offset and flag_time_offset_acquired to 0.
When flag_time_offset_acquired == 0, bypass wait_for_frame() function in both decode_video() and write_audio().
When flag_time_offset_acquired == 0, use the estimated time (I copied below to show which time I am referring to).


    int64_t pts = av_frame_get_best_effort_timestamp(frame);
    if (pts == AV_NOPTS_VALUE) {
        pts = 0;
    }
    int64_t time = av_rescale_q(pts, stream->time_base, AV_TIME_BASE_Q); 

        // New code here.
        if (player->flag_time_offset_acquired == 0 &&   pts != AV_NOPTS_VALUE) {
        pthread_mutex_lock(&player->mutex_queue);
            player->start_time = av_gettime();
        player->pause_time = player->start_time;
            player->time_offset = time;
            player->flag_time_offset_acquired = 1;
        pthread_mutex_unlock(&player->mutex_queue);
        }

In jni_player_resume(),


    if (player->flag_time_offset_acquired == 1) {
        player->start_time += resume_time - player->pause_time;
    }
    else {
        player->start_time = resume_time;
        player->pause_time = resume_time;
    }

Last step is inside wait_for_frame(), the calculation of "sleep_time" is changed to this.

        int64_t sleep_time = stream_time - player->time_offset - current_video_time;

With the above, stream_time - player->time_offset is the stream playing duration while current_video_time is the difference between av_gettime() - player->start_time with player->start_time initialized to av_gettime() at the moment when player->time_offset is initialized. In this way, sleep_time can fully reflect the time playing difference in streaming and in local time.

Make sense?

appunite / AndroidFFmpeg

Player stuck in wait_for_frame() in playing an RTMP link #128