transcode example doesn't work with h264

[voxeljorge]

Hi,

In the transcode example, transcoding fails if you change the encoder to astiav.CodecIDH264

I'm on the latest release, with go 1.23.2.

Here's the end of the output:

2024/11/13 01:49:50 ffmpeg log: Decoding SPS - class: hevc [AVCodecContext] @ 0x4a3ab80 - level: 48
2024/11/13 01:49:50 ffmpeg log: Main profile bitstream - class: hevc [AVCodecContext] @ 0x4a3ab80 - level: 48
2024/11/13 01:49:50 ffmpeg log: Decoding VUI - class: hevc [AVCodecContext] @ 0x4a3ab80 - level: 48
2024/11/13 01:49:50 ffmpeg log: nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0 - class: hevc [AVCodecContext] @ 0x4a3ab80 - level: 48
2024/11/13 01:49:50 ffmpeg log: Decoding PPS - class: hevc [AVCodecContext] @ 0x4a3ab80 - level: 48
2024/11/13 01:49:50 ffmpeg log: using mv_range_thread = 24 - class: libx264 [AVCodecContext] @ 0x4a37500 - level: 48
2024/11/13 01:49:50 ffmpeg log: using SAR=1/1 - class: libx264 [AVCodecContext] @ 0x4a37500 - level: 32
2024/11/13 01:49:50 ffmpeg log: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512 - class: libx264 [AVCodecContext] @ 0x4a37500 - level: 32
2024/11/13 01:49:50 ffmpeg log: profile High, level 3.1, 4:2:0, 8-bit - class: libx264 [AVCodecContext] @ 0x4a37500 - level: 32
2024/11/13 01:49:50 ffmpeg log: Setting default whitelist 'file,crypto,data' - class: file [URLContext] @ 0x4b86700 - level: 48
2024/11/13 01:49:50 ffmpeg log: get_metadata_duration returned: 0 - class: matroska [AVFormatContext] @ 0x4a36a00 - level: 48
2024/11/13 01:49:50 main: opening output file failed: main: writing header failed: Invalid data found when processing input

I tried making some changes by setting various parameters but that didn't seem to work. I wonder if some setting needs to be set that is not exposed by the library?

[voxeljorge]

Forcing global headers seems to get it slightly further, but it fails on the first packet:

2024/11/13 07:19:36 ffmpeg log: Decoding SEI - class: hevc [AVCodecContext] @ 0x4828b80 - level: 48
2024/11/13 07:19:36 ffmpeg log: Output frame with POC 8. - class: hevc [AVCodecContext] @ 0x4828b80 - level: 48
2024/11/13 07:19:36 ffmpeg log: Decoded frame with POC 10. - class: hevc [AVCodecContext] @ 0x4828b80 - level: 48
2024/11/13 07:19:36 ffmpeg log: Changing video frame properties on the fly is not supported by all filters. - class: in [AVFilter] @ 0x4bda040 - level: 48
2024/11/13 07:19:36 ffmpeg log: filter context - w: 960 h: 720 fmt: 0 csp: unknown range: unknown, incoming frame - w: 960 h: 720 fmt: 0 csp: unknown range: tv pts_time: NOPTS - class: in [AVFilter] @ 0x4bda040 - level: 48
2024/11/13 07:19:36 ffmpeg log: non-strictly-monotonic PTS - class: libx264 [AVCodecContext] @ 0x4825500 - level: 24
2024/11/13 07:19:36 ffmpeg log: frame=   0 QP=25.05 NAL=3 Slice:I Poc:0   I:2700 P:0    SKIP:0    size=52506 bytes - class: libx264 [AVCodecContext] @ 0x4825500 - level: 48
2024/11/13 07:19:36 ffmpeg log: Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly - class: matroska [AVFormatContext] @ 0x4824a00 - level: 24
2024/11/13 07:19:36 main: filtering, encoding and writing frame failed: main: encoding and writing frame failed: main: writing frame failed: Invalid argument```

[asticode]

non-strictly-monotonic PTS + Timestamps are unset in a packet for stream 0: there seems to have problems with the PTS provided to the encoder then the muxer since they seem to be unset.

What is your input?

[voxeljorge]

It's just a sample mkv pulled off a camera. Pretty sure that warning is just a warning but I can try with better input. The CLI gives the same warning but still transcodes the file.

[voxeljorge]

I get the same error with a file generated using the testsrc:

ffmpeg -f lavfi -i testsrc -t 30 -c:v hevc testsrc.mkv

Haven't been able to get past this for a h264 encode, still getting the same error:

2024/11/13 21:04:01 pkt pts=2080 dts=2000
2024/11/13 21:04:01 ffmpeg log: nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0 - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Output frame with POC 50. - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Decoded frame with POC 52. - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Changing video frame properties on the fly is not supported by all filters. - class: in [AVFilter] @ 0x4a30040 - level: 48
2024/11/13 21:04:01 ffmpeg log: filter context - w: 320 h: 240 fmt: 71 csp: unknown range: unknown, incoming frame - w: 320 h: 240 fmt: 71 csp: gbr range: tv pts_time: NOPTS - class: in [AVFilter] @ 0x4a30040 - level: 48
2024/11/13 21:04:01 ffmpeg log: non-strictly-monotonic PTS - class: libx264 [AVCodecContext] @ 0x4234680 - level: 24
2024/11/13 21:04:01 pkt pts=2040 dts=2040
2024/11/13 21:04:01 ffmpeg log: nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0 - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Output frame with POC 51. - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Decoded frame with POC 51. - class: hevc [AVCodecContext] @ 0x41252c0 - level: 48
2024/11/13 21:04:01 ffmpeg log: Changing video frame properties on the fly is not supported by all filters. - class: in [AVFilter] @ 0x4a30040 - level: 48
2024/11/13 21:04:01 ffmpeg log: filter context - w: 320 h: 240 fmt: 71 csp: unknown range: unknown, incoming frame - w: 320 h: 240 fmt: 71 csp: gbr range: tv pts_time: NOPTS - class: in [AVFilter] @ 0x4a30040 - level: 48
2024/11/13 21:04:01 ffmpeg log: non-strictly-monotonic PTS - class: libx264 [AVCodecContext] @ 0x4234680 - level: 24
2024/11/13 21:04:01 ffmpeg log: frame=   0 QP=23.58 NAL=3 Slice:I Poc:0   I:300  P:0    SKIP:0    size=2787 bytes - class: libx264 [AVCodecContext] @ 0x4234680 - level: 48
2024/11/13 21:04:01 ffmpeg log: Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly - class: matroska [AVFormatContext] @ 0x4167500 - level: 24
2024/11/13 21:04:01 main: filtering, encoding and writing frame failed: main: encoding and writing frame failed: main: writing frame failed: Invalid argument

[voxeljorge]

I think I've figured out the issue. The decCodecContext in the example does not set decCodecContext.TimeBase() even though this time base is used in the decoder to rescale timestamps from input packets like this: pkt.RescaleTs(s.inputStream.TimeBase(), s.decCodecContext.TimeBase())

So I would say there are two small-ish bugs in the example:

1) Whether or not to set global headers is more dependent on the output format than the input format, but the example sets global headers based on whether the input format has that flag set. 2) the decoder context doesn't set TimeBase even though it does use TimeBase in this example

[asticode]

Nice catches 👍

Whether or not to set global headers is more dependent on the output format than the input format, but the example sets global headers based on whether the input format has that flag set.

👍

the decoder context doesn't set TimeBase even though it does use TimeBase in this example

Are you sure it fixes the issue? 🤔 After testing very quickly, error disappeared but it resulted with a video in slow motion 🤔

[voxeljorge]

Are you sure it fixes the issue? 🤔 After testing very quickly, error disappeared but it resulted with a video in slow motion 🤔

That likely means one of the time base values is set incorrectly.

I've also encountered one more issue trying to encode h265 video:

2024/11/14 22:37:22 ffmpeg log: filter context - w: 320 h: 240 fmt: 71 csp: unknown range: unknown, incoming frame - w: 320 h: 240 fmt: 71 csp: gbr range: tv pts_time: 0.027 - class: in [AVFilter] @ 0x44f4400 - level: 48
2024/11/14 22:37:22 ffmpeg log: Unknown picture type encountered. - class: libx265 [AVCodecContext] @ 0x4307680 - level: 16
2024/11/14 22:37:22 main: filtering, encoding and writing frame failed: main: encoding and writing frame failed: main: sending frame failed: Generic error in an external library

The relevant error seems to be "Unknown picture type encountered"

I've tried fiddling with the picture type of the frames and can't seem to find any setting that resolves this error. Looking at the source code I don't really see how this could be set incorrectly as it should be set to an auto value anyway.