search

appunite / AndroidFFmpeg

[DEPRECATED] FFmpeg build for android random architectures with example jni
Apache License 2.0
1.07k stars 471 forks source link

Stream frame timestamp problem #73

Open 4ntoine opened 10 years ago

4ntoine commented 10 years ago

I've compiled ffmpeg-android and it works for files. The problem is that it shows nothing for network streams (both rtmp and hls) as frame timestamp is too big and it seems to be not from video beginning (as for file). Then it waits for the first frame time which for network stream is too big:

player_decode_video copying... 05-05 18:11:26.994: INFO/player.c(16998): player_decode_video Decoded video frame: 568.233000, time_base: 51140970 05-05 18:11:26.994: INFO/player.c(16998): player_wait_for_frame[0] start ... 05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.000977) 05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 568232023 05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] timeout 05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.501542) 05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567731458 05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] timeout 05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.002778) 05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567230222 05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] timeout 05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.504563) 05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566728437 05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] timeout 05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (2.005708) 05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566227292

4ntoine commented 10 years ago

also i can't figure out what timestamp for frame for network stream means (for file frames it's a time from video beginning)

4ntoine commented 10 years ago

for file it's clear: time_base is frame number and timestamp is time from video beginning

player_decode_video Decoded video frame: 0.320000, time_base: 8 05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] start 05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.320000) - (0.344337) 05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -24337 05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] finish[0] 05-05 18:32:42.344: INFO/player.c(16998): player_update_time: 0.346169/4.000000 05-05 18:32:42.344: INFO/player.c(16998): player_decode waiting for frame[0] 05-05 18:32:42.344: INFO/player.c(16998): player_decode decoding frame[0] 05-05 18:32:42.344: INFO/player.c(16998): player_decode_video decoding 05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copy wait 05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream Read frame 05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream looking for stream 05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream stream found [0] 05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream waiting for queue 05-05 18:32:42.344: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888 05-05 18:32:42.344: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280 05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copying... 05-05 18:32:42.425: INFO/player.c(16998): player_decode_video Decoded video frame: 0.360000, time_base: 9 05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] start 05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.360000) - (0.427994) 05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -67994 05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] finish[0] 05-05 18:32:42.425: INFO/player.c(16998): player_update_time: 0.429214/4.000000 05-05 18:32:42.425: INFO/player.c(16998): player_decode waiting for frame[0] 05-05 18:32:42.425: INFO/player.c(16998): player_decode decoding frame[0] 05-05 18:32:42.425: INFO/player.c(16998): player_decode_video decoding 05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream Read frame 05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream looking for stream 05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream stream found [0] 05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream waiting for queue 05-05 18:32:42.425: INFO/player.c(16998): player_decode_video copy wait 05-05 18:32:42.435: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888 05-05 18:32:42.435: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280 05-05 18:32:42.435: INFO/player.c(16998): player_decode_video copying... 05-05 18:32:42.495: INFO/player.c(16998): player_decode_video Decoded video frame: 0.400000, time_base: 10 05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] start 05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.400000) - (0.494742) 05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -94742 05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] finish[0] 05-05 18:32:42.495: INFO/player.c(16998): player_update_time: 0.495993/4.000000 05-05 18:32:42.495: INFO/player.c(16998): player_decode waiting for frame[0] 05-05 18:32:42.495: INFO/player.c(16998): player_decode decoding frame[0] 05-05 18:32:42.495: INFO/player.c(16998): player_decode_video decoding 05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copy wait 05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream Read frame 05-05 18:32:42.495: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888 05-05 18:32:42.495: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280 05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copying... 05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream looking for stream 05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream stream found [0] 05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream waiting for queue 05-05 18:32:42.555: INFO/player.c(16998): player_decode_video Decoded video frame: 0.440000, time_base: 11 05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] start 05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.440000) - (0.556698) 05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -116698 05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] finish[0] 05-05 18:32:42.555: INFO/player.c(16998): player_update_time: 0.557858/4.000000 05-05 18:32:42.555: INFO/player.c(16998): player_decode waiting for frame[0] 05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream Read frame 05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream looking for stream 05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream stream found [0] 05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream waiting for queue 05-05 18:32:42.555: INFO/player.c(16998): player_decode decoding frame[0] 05-05 18:32:42.555: INFO/player.c(16998): player_decode_video decoding 05-05 18:32:42.555: INFO/player.c(16998): player_decode_video copy wait 05-05 18:32:42.565: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888 05-05 18:32:42.565: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280 05-05 18:32:42.565: INFO/player.c(16998): player_decode_video copying... 05-05 18:32:42.625: INFO/player.c(16998): player_decode_video Decoded video frame: 0.480000, time_base: 12

nxtreaming commented 10 years ago

The following codes are wrong:

inline int64_t player_get_current_video_time(struct Player *player) { if (player->pause) { return player->pause_time - player->start_time; } else { int64_t current_time = av_gettime(); return current_time - player->start_time; } }

A/V sync should not depend on local time.

4ntoine commented 10 years ago

I can't see any reason for it for the files to be wrong as time is relative to video beginning. How should this be changed for playing stream (rtmp/hls) ?

Joboodi commented 8 years ago

I'm having this problem too. What happens is that the Video or Audio stream decoding will decode a frame who says it's frame time is wildly different from the previous frame and it will stay that way for every subsequent decoded frame (just like the log that @4ntoine posted. I wrote some code that would detect when this happened and offset the frame start time by that same wildly different delta. If I applied the same code to @4ntoine 's log, it would end up looking like this instead...

05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (528.200977)

and thus the video frame would only delay for a few milliseconds or so and the stream would continue playing. However, there is a problem with this "fix" (hack). It causes the video and the audio to get out of sync because that offset that I apply to the stream that's gone bad has still altered the start frame offset from what it originally was - thus having the effect of causing the frame to show earlier or later than it normally would (if the frame wasn't reporting a horribly wrong frame time, that is).

Plus, the offset I add in to correct the wildly wrong frame time is just a guess. I take the delta of the av_getTime() from the previous frame with the av_getTime() of the current frame. That tells me how much time has really passed. Then I subtract that out of the wildly wrong frame time of the frame to get my start offset delta that needs to be added to the stream start frame time. That bumps up the current time to match (roughly) the current time of the messed up frame. But like I said, that calculated value is assuming that the messed-up frame's time wants it to render when I'm doing the calculation without some natural offset, which is never the case. Thus, like I said, all I've really done is to offset video playback slightly and put it out of sync from the audio.

If there are better (real) solutions to this problem I would be excited to hear them. As things stand currently this is a deal-breaker issue for me to be able to play video from IP cameras.