GetTime() always returns 0 when the video has no audio track

[jasonchin]

I am using UE 4.22 and have tested using H265 and H264 videos.

If the video file does not contain an audio track, the value returned from FFMPEGMediaTracks::GetTime() (called via UE4's MediaPlayer::GetTime() interface) is always 0.

It seems to stem from the below line: https://github.com/bakjos/FFMPEGMedia/blob/161c5d1acb42f1aae0708b3f03abb638f58d2425/Source/FFMPEGMedia/Private/Player/FFMPEGMediaTracks.cpp#L2505

I haven't looked into it that much yet but it seems like vp->GetPts() is returning incorrect values.

[bakjos]

@jasonchin does the file contains the audio stream even with no sound? if so, I think that's the issue, because the plugin will try to sync the audio and video streams. You can check the streams using ffmpeg ffmpeg -i <input> 2>&1 | grep "Stream #" | wc -l you can remove the audio stream using the instructions from here or use any video edition software.

If the file doesn't have a sound stream, can you share the file with me? so I can test locally?

[jasonchin]

@bakjos Here is a sample H265 video file: FrameCounterTest-H265.zip

It definitely doesn't have an audio stream, I have stripped it out using -an in ffmpeg.

For clarity here is the command I use to generate the files: ffmpeg.exe -i <input> -c:v libx265 -an <output>

[bakjos]

@jasonchin Thanks for upload the video, I made a couple of tests and the video is playing correctly for me. Since the format can be HW accelerated, sometimes it will try to use your GPU to play the video, you can go to the project settings and inside the plugins section and disable the Use hardware accelerated codecs option

What does the ouput log show? is it showing anything about the driver something like

LogFFMPEGMedia: Error: FFMPEG - Could not initialize the CUDA driver API or LogFFMPEGMedia: Error: FFMPEG - Failed to create Direct3D device ?

Can you send me the full output log?

[jasonchin]

@bakjos Thanks for your response.

Thanks for upload the video, I made a couple of tests and the video is playing correctly for me.

To be clear, the video played fine previously, but just that GetTime() always returned 0.

Since the format can be HW accelerated, sometimes it will try to use your GPU to play the video, you can go to the project settings and inside the plugins section and disable the Use hardware accelerated codecs option

This seems to have solved my problem, thanks for the suggestion. With HW acceleration disabled, the correct frame number is returned. Does that mean that with HW acceleration enabled, getting the correct frame number is not possible? Or do you think that this is HW dependant issue?

Can you send me the full output log?

Below is the log for when HW accel is enabled. There aren't any FFMPEG errors. The LogBlueprintUserMessages: [Level_C_1] 0.0 outputs are me calling GetTime() every tick.

FFMPEGMedia-issue4log.txt

[bakjos]

The issue is the LogFFMPEGMedia: Warning: FFMPEG - Invalid pkt_timebase, passing timestamps as-is., this is returned directly inside the cuda operation https://github.com/FFmpeg/FFmpeg/blob/19f75e77870ac8bca06d379c2ac2c3d652e6d08b/libavcodec/cuviddec.c#L1051. Can you try using an external clock sync type (The last option in the plugin settings) and enabling the HW acceleration?

[jasonchin]

Unfortunately it does not work, GetTime() is still returning 0.

Here is the log: FFMPEGMedia-issue4log-2.txt

[jasonchin]

@bakjos Hi, I've noticed that this issue even happens even with ffplay when using a HW accelerated codec. The frame time is not returned, so I suspect it is an issue within the codec itself.

eg .\ffplay.exe [file] -vcodec hevc_cuvid will display 'nan' for the frame time.

This is unfortunate, but doesn't seem like it can be easily fixed. I will close this ticket for now (feel free to open it if you have any other ideas). I would suggest making a note of it somewhere in the documentation/comments that this is an issue to look out for (although I appreciate that could be difficult with all the many codecs that ffmpeg supports).

[bakjos]

@jasonchin Thanks for looking into it, it might get fixed in newer versions of FFMPEG