H.264 Bitstream malformed, possible error in ffmpeg settings or upscaler.py?

[andyhines]

I have run into this error using the waifu2x-caffe driver on 4.1.0 and 4.2.0, but not on 4.0.0 beta-3. I tested multiple input files and each seems to hit the same error, and I tested files that I previously had successfully used as inputs to 4.0.0b3 and they too failed, so I suspect that this is not specific to the files (or if it is it's an issue with all of my test files, that previously was handled smoothly by 4.0.0b3). I switched back to 4.0.0b3 and have successfully upscaled a file that gave this error so I don't suspect it is inherent to any recent driver changes.

It looks like perhaps there was something changed in the way ffmpeg is called? The video upscaling seems to actually complete and creates a video file output with no audio and a second file (that I didn't check but I suspect contains the audio stream) in the output folder, it just then fails to combine them, and stops processing any other input files (due to the error.) It seems odd that it would create a malformed video file that it then couldn't re-integrate into the final output but perhaps I am misunderstanding the error.

Thank you for any advice!

[k4yt3x]

I will look into this.

[k4yt3x]

Do you mind sending me the original footage for me to take a look?

[k4yt3x]

I was reading this post about what this problem is and how to fix it: https://blog.csdn.net/IOT_SHUN/article/details/79739209 (zh_CN) The solution is actually also provided in the output. You can add -bsf h264_mp4toannexb to the stream migration command to fix the problem.

[k4yt3x]

Then your config should look something like this

[andyhines]

Ah good call! I will give that a shot! I feel foolish for not simply trying it after reading through the error, but I figured it was worth letting you know and seeing if anyone else had this trouble. I can send you a file if you need, but i'm sure you're busy so for now I will test and see if this fixes it! Either way I will let you know. Thank you!

[andyhines]

I did add that parameter and tested on a video, unfortunately it did still error out on me. Slightly different now, no indication (that I see) of a new parameter to add to fix it. I dropped the file in question into the FFprobe tool you added (thank you!) and here's the output:

{ "streams": [ { "index": 0, "codec_name": "mpeg4", "codec_long_name": "MPEG-4 part 2", "codec_type": "video", "codec_time_base": "125/2997", "codec_tag_string": "XVID", "codec_tag": "0x44495658", "width": 640, "height": 480, "coded_width": 640, "coded_height": 480, "closed_captions": 0, "has_b_frames": 0, "sample_aspect_ratio": "1:1", "display_aspect_ratio": "4:3", "pix_fmt": "yuv420p", "level": -99, "chroma_location": "left", "refs": 1, "quarter_sample": "false", "divx_packed": "false", "r_frame_rate": "2997/125", "avg_frame_rate": "2997/125", "time_base": "125/2997", "start_pts": 0, "start_time": "0.000000", "duration_ts": 33428, "duration": "1394.227561", "bit_rate": "847548", "nb_frames": "33428", "disposition": { "default": 0, "dub": 0, "original": 0, "comment": 0, "lyrics": 0, "karaoke": 0, "forced": 0, "hearing_impaired": 0, "visual_impaired": 0, "clean_effects": 0, "attached_pic": 0, "timed_thumbnails": 0 } }, { "index": 1, "codec_name": "ac3", "codec_long_name": "ATSC A/52A (AC-3)", "codec_type": "audio", "codec_time_base": "1/48000", "codec_tag_string": "[0] [0][0]", "codec_tag": "0x2000", "sample_fmt": "fltp", "sample_rate": "48000", "channels": 2, "channel_layout": "stereo", "bits_per_sample": 0, "dmix_mode": "-1", "ltrt_cmixlev": "-1.000000", "ltrt_surmixlev": "-1.000000", "loro_cmixlev": "-1.000000", "loro_surmixlev": "-1.000000", "r_frame_rate": "0/0", "avg_frame_rate": "0/0", "time_base": "1/24000", "start_pts": 0, "start_time": "0.000000", "bit_rate": "192000", "nb_frames": "33461160", "disposition": { "default": 0, "dub": 0, "original": 0, "comment": 0, "lyrics": 0, "karaoke": 0, "forced": 0, "hearing_impaired": 0, "visual_impaired": 0, "clean_effects": 0, "attached_pic": 0, "timed_thumbnails": 0 } } ], "format": { "filename": "F:\VideoUpscaling\Ranma\Season 03\Ranma_s03e13.avi", "nb_streams": 2, "nb_programs": 0, "format_name": "avi", "format_long_name": "AVI (Audio Video Interleaved)", "start_time": "0.000000", "duration": "1394.227561", "size": "182345728", "bit_rate": "1046289", "probe_score": 100 } }

Apologies for the wall of text! If it helps, here the file input that caused this issue.

The output folder produced a .mkv video file that has no audio, and a 0 byte .avi, so (to me) it seems it created the container then just failed to push the .mkv into it. Strange! I have since upscaled a few more videos on beta 4.0.0b3 and I do not know why that one works on these files.

Thanks for your help!

[k4yt3x]

The audio stream in your original file is in ac3 codec, which is not supported by MP4 encap format. Uncheck the stream copy option and FFmpeg will choose a codec automatically for the new container format.

[dgcampea]

Try:

• Keep copy streams on (contrary to above post)
• Instead of -bsf h264_mp4toannexb, use -bsf:v h264_mp4toannexb (Note the :v).

[k4yt3x]

or change output to MKV

[andyhines]

Ah, will test that. Thank you!

[andyhines]

Using -bsf:v h264_mp4toannexb did solve the issue, I was able to get upscaling to complete just fine. I will use those settings from now on! Closing for now, thank you!