How to encode I-Frames, video seeking and other misc. issues

[shroom00]

As far as I know, I-frames (sync frames?) are used for seeking. However, only P frames are being written. Assuming that this is true, why are videos made using the following code unable be seeked through? By this, I mean that the video plays from the beginning fine but can't be resumed from mid-way (when this is attempted, it starts over from the beginning). Technically, seeking works in VLC, but as it doesn't work in my editor (that does support seeking), I think this is due to some VLC-specific implementation details.

main.rs.txt

The dependencies for the code are as follows:

image = "0.25.2"
mp4 = "0.14.0"
openh264 = "0.6.1"

Running the command ffprobe -v error -show_format -show_streams out.mp4, (ffprobe is bundled with ffmpeg) the following errors appear:

[h264 @ 000001b7db0e7b00] Overread VUI by 8 bits
[h264 @ 000001b7db0e7b00] Invalid NAL unit size (1119884428 > 45).
[h264 @ 000001b7db0e7b00] missing picture in access unit with size 54

How would I resolve these? What could be causing them to happen?

Running the command ffmpeg -i out.mp4 -c copy -bsf:v trace_headers -f null - also indicates that the h264 profile of the video is "Constrained Baseline"; This is the same, regardless of the profile/level specified in the SPS buffer.

I'm not sure if all this is more appropriate to the mp4 crate or openh264, but as there are h264 specific things, I figured I'll try here. Any help/pointers are appreciated. :)

[ralfbiedert]

I don't have any experience with video seeking behavior, but it'd be helpful if you either post or link a video with the issue, or use a H264 Analyzer to see for yourself what's going on in the file. Also, could you post your code how you generated said file?

[ralfbiedert]

Alright, I just noticed you attached the code as a .txt file. I gave this a quick look and wrote a .H264 file instead (by concatenating the bitstream results). The result looks like so:

Notably, the SPS and PPS information differs from the one you specify manually:

SPS:

==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 3
nal_unit_type : 7 ( Sequence parameter set )
======= SPS =======
profile_idc : 66
constraint_set0_flag : 1
constraint_set1_flag : 1
constraint_set2_flag : 0
constraint_set3_flag : 0
constraint_set4_flag : 0
constraint_set5_flag : 0
reserved_zero_2bits : 0
level_idc : 31
seq_parameter_set_id : 0
chroma_format_idc : 1
residual_colour_transform_flag : 0
bit_depth_luma_minus8 : 0
bit_depth_chroma_minus8 : 0
qpprime_y_zero_transform_bypass_flag : 0
seq_scaling_matrix_present_flag : 0
log2_max_frame_num_minus4 : 11
pic_order_cnt_type : 0
log2_max_pic_order_cnt_lsb_minus4 : 12
delta_pic_order_always_zero_flag : 0
offset_for_non_ref_pic : 0
offset_for_top_to_bottom_field : 0
num_ref_frames_in_pic_order_cnt_cycle : 0
num_ref_frames : 1
gaps_in_frame_num_value_allowed_flag : 0
pic_width_in_mbs_minus1 : 49
pic_height_in_map_units_minus1 : 29
frame_mbs_only_flag : 1
mb_adaptive_frame_field_flag : 0
direct_8x8_inference_flag : 1
frame_cropping_flag : 1
frame_crop_left_offset : 0
frame_crop_right_offset : 0
frame_crop_top_offset : 0
frame_crop_bottom_offset : 5
vui_parameters_present_flag : 1
=== VUI ===
aspect_ratio_info_present_flag : 0
aspect_ratio_idc : 0
sar_width : 0
sar_height : 0
overscan_info_present_flag : 0
overscan_appropriate_flag : 0
video_signal_type_present_flag : 0
video_format : 0
video_full_range_flag : 0
colour_description_present_flag : 0
colour_primaries : 0
transfer_characteristics : 0
matrix_coefficients : 0
chroma_loc_info_present_flag : 0
chroma_sample_loc_type_top_field : 0
chroma_sample_loc_type_bottom_field : 0
timing_info_present_flag : 0
num_units_in_tick : 0
time_scale : 0
fixed_frame_rate_flag : 0
nal_hrd_parameters_present_flag : 0
vcl_hrd_parameters_present_flag : 0
low_delay_hrd_flag : 0
pic_struct_present_flag : 0
bitstream_restriction_flag : 1
motion_vectors_over_pic_boundaries_flag : 1
max_bytes_per_pic_denom : 0
max_bits_per_mb_denom : 0
log2_max_mv_length_horizontal : 16
log2_max_mv_length_vertical : 16
num_reorder_frames : 0
max_dec_frame_buffering : 1
=== HRD ===
cpb_cnt_minus1 : 0
bit_rate_scale : 0
cpb_size_scale : 0
bit_rate_value_minus1[0] : 0
cpb_size_value_minus1[0] : 0
cbr_flag[0] : 0
initial_cpb_removal_delay_length_minus1 : 0
cpb_removal_delay_length_minus1 : 0
dpb_output_delay_length_minus1 : 0
time_offset_length : 0 

PPS:

==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 3
nal_unit_type : 8 ( Picture parameter set )
======= PPS =======
pic_parameter_set_id : 0
seq_parameter_set_id : 0
entropy_coding_mode_flag : 0
pic_order_present_flag : 0
num_slice_groups_minus1 : 0
slice_group_map_type : 0
num_ref_idx_l0_active_minus1 : 0
num_ref_idx_l1_active_minus1 : 0
weighted_pred_flag : 0
weighted_bipred_idc : 0
pic_init_qp_minus26 : 0
pic_init_qs_minus26 : 0
chroma_qp_index_offset : 0
deblocking_filter_control_present_flag : 1
constrained_intra_pred_flag : 0
redundant_pic_cnt_present_flag : 0
transform_8x8_mode_flag : 0
pic_scaling_matrix_present_flag : 0
second_chroma_qp_index_offset : 0 

Also, you probably should double check how your container format wants these (e.g., 001 prefixed vs u32 prefixed, ...). I'd try to avoid setting these manually if you can. I'm not saying this is why it gives you seeking issues, but this is where I'd start looking.

[ralfbiedert]

Closing for now, this seems to be unrelated to OpenH264. If this turns out to be a bug on our side, feel free to reopen.