Upscaling `.bik` Video with h264_nvenc Fails

[ThisNekoGuy]

System Info

OS: Gentoo Linux GPU: Nvidia RTX 2080 Ti GPU Driver: 535.104.05

Description

I was trying to upscale cutscene .bik videos for an old game (namely Splinter Cell 1 from Steam), and video2x seems to be throwing an error related to ffmpeg, somehow, and it's unclear to me what exactly the problem is when reading the log (and there also appears to be several things going wrong - such as a missing streams?)...

Ideally, for context, I also need the output to be back in the .bik format but it doesn't seem to be possible with ffmpeg(? in a prior test), so I opted to output it to mkv until I can find a way to convert it back (but that's a bit off-topic from the main issue here).

Log and Config File

video2x_uhfd72tx.log

video2x.yaml configuration file:

```yaml # Name: Video2X Configuration File # Values here are the default values. Change the value here to # save the default value permanently. # Items commented out are parameters irrelevant to this context # or parameters handled by Video2X internally. waifu2x_caffe: path: '/usr/bin/waifu2x-caffe-cui' tta: 0 # <0|1> 8x slower and slightly high quality gpu: 0 # gpu device no batch_size: 1 # input batch size crop_h: null # input image split size(height) crop_w: null # input image split size(width) crop_size: 128 # input image split size output_depth: 8 # output image chanel depth bit output_quality: -1 # output image quality process: gpu # process mode model_dir: null # path to custom model directory (don't append last / ) #scale_height: 0 # custom scale height (specifying this will overwrite scale_ratio) #scale_width: 0 # custom scale width (specifying this will overwrite scale_ratio) #scale_ratio: null # custom scale ratio noise_level: 3 # <0|1|2|3> noise reduction level mode: noise_scale # image processing mode output_extention: null # extension to output image file when output_path is (auto) or input_path is folder input_extention_list: null # extension to input image file when input_path is folder #output_path: null # path to output image file (when input_path is folder, output_path must be folder) #input_path: null # (required) path to input image file waifu2x_converter_cpp: path: '/usr/bin/waifu2x-converter-cpp' #list-supported-formats: null # dump currently supported format list #list-opencv-formats: null # (deprecated. Use --list-supported-formats) dump opencv supported format list #list-processor # dump processor list #output-format: null # The format used when running in recursive/folder mode png-compression: 5 # Set PNG compression level (0-9), 9 = Max compression (slowest & smallest) image-quality: -1 # JPEG & WebP Compression quality (-1-101, 0 being smallest size and lowest quality, -1 being default), use 101 for lossless WebP block-size: 0 # block size disable-gpu: false # disable GPU force-OpenCL: false # force to use OpenCL on Intel Platform processor: -1 # set target processor (-1 uses default device) jobs: 0 # number of threads launching at the same time model-dir: null # path to custom model directory (don't append last / ) default: models_rgb #scale-ratio: 2.0 # custom scale ratio noise-level: 1 # <0|1|2|3> noise reduction level mode: noise-scale # image processing mode log-level: 1 # <0|1|2|3|4> Set log level silent: true # Enable silent mode. (same as --log-level 1) tta: 0 # Enable Test-Time Augmentation mode. (0 or 1) #generate-subdir: 0 # Generate sub folder when recursive directory is enabled. #auto-naming: 0 # Add postfix to output name when output path is not specified. #recursive-directory: 0 # Search recursively through directories to find more images to process. #output: null # path to output image file or directory (you should use the full path) #input: null # (required) path to input image file or directory (you should use the full path) waifu2x_ncnn_vulkan: path: '/usr/bin/waifu2x-ncnn-vulkan' v: null # verbose output #i: null # input-path: input image path (jpg/png/webp) or directory #o: null # output-path: output image path (jpg/png/webp) or directory 'n': 3 # noise-level: denoise level (-1/0/1/2/3, default=0) s: 2 # scale: upscale ratio (1/2, default=2) t: 400 # tile-size: tile size (>=32, default=400) m: null # model-path: waifu2x model path (default=models-cunet) g: 0 # gpu-id: gpu device to use (default=0) j: '1:2:2' # thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu x: false # enable tta mode #f: png # output image format (jpg/png/webp, default=ext/png) srmd_ncnn_vulkan: path: '/usr/bin/srmd-ncnn-vulkan' v: null # verbose output #i: null # input-path: input image path (jpg/png) or directory #o: null # output-path: output image path (png) or directory 'n': 3 # noise-level: denoise level (-1/0/1/2/3/4/5/6/7/8/9/10, default=3) s: 2 # upscale ratio (2/3/4, default=2) t: 400 # tile-size: tile size (>=32, default=400) m: null # srmd model path (default=models-srmd) g: 0 # gpu device to use (default=0) j: '1:2:2' # thread count for load/proc/save (default=1:2:2) x: false # enable tta mode #f: png # output image format (jpg/png/webp, default=ext/png) realsr_ncnn_vulkan: path: '/usr/bin/realsr-ncnn-vulkan' v: null # verbose output #i: null # input-path: input image path (jpg/png) or directory #o: null # output-path: output image path (png) or directory s: 4 # upscale ratio (4, default=4) t: 0 # tile size (>=32/0=auto, default=0) m: null # realsr model path (default=models-DF2K) g: 0 # gpu device to use (default=0) j: '1:2:2' # thread count for load/proc/save (default=1:2:2) x: false # enable tta mode #f: png # output image format (jpg/png/webp, default=ext/png) anime4kcpp: path: '/usr/bin/Anime4KCPP_CLI' #input: null # File for loading (string [=./pic/p1.png]) #output: null # File for outputting (string [=output.png]) passes: 2 # Passes for processing (int [=2]) pushColorCount: 2 # Limit the number of color pushes (int [=2]) strengthColor: 0.3 # Strength for pushing color,range 0 to 1,higher for thinner (double [=0.3]) strengthGradient: 1.0 # Strength for pushing gradient,range 0 to 1,higher for sharper (double [=1]) zoomFactor: 2.0 # zoom factor for resizing (double [=2]) threads: 16 # Threads count for video processing (unsigned int [=16]) fastMode: false # Faster but maybe low quality videoMode: false # Video process preview: null # Preview image preprocessing: false # Enable pre processing postprocessing: false # Enable post processing preFilters: 4 # Enhancement filter, only working when preProcessing is true,there are 5 options by binary:Median blur=0000001, Mean blur=0000010, CAS Sharpening=0000100, Gaussian blur weak=0001000, Gaussian blur=0010000, Bilateral filter=0100000, Bilateral filter faster=1000000, you can freely combine them, eg: Gaussian blur weak + Bilateral filter = 0001000 | 0100000 = 0101000 = 40(D) (unsigned int [=4]) postFilters: 40 # Enhancement filter, only working when postProcessing is true,there are 5 options by binary:Median blur=0000001, Mean blur=0000010, CAS Sharpening=0000100, Gaussian blur weak=0001000, Gaussian blur=0010000, Bilateral filter=0100000, Bilateral filter faster=1000000, you can freely combine them, eg: Gaussian blur weak + Bilateral filter = 0001000 | 0100000 = 0101000 = 40(D), so you can put 40 to enable Gaussian blur weak and Bilateral filter, which also is what I recommend for image that < 1080P, 48 for image that >= 1080P, and for performance I recommend to use 72 for video that < 1080P, 80 for video that >=1080P (unsigned int [=40]) GPUMode: false # Enable GPU acceleration CNNMode: false # Enable ACNet HDN: false # Enable HDN mode for ACNet HDNLevel: 1 # Set HDN level (int [=1]) listGPUs: null # list GPUs platformID: 0 # Specify the platform ID (unsigned int [=0]) deviceID: 0 # Specify the device ID (unsigned int [=0]) codec: avc1 # Specify the codec for encoding from mp4v(recommended in Windows), dxva(for Windows), avc1(H264, recommended in Linux), vp09(very slow), hevc(not support in Windowds), av01(not support in Windowds) (string [=mp4v]) forceFps: 0 # Set output video fps to the specifying number, 0 to disable (float [=0]) disableProgress: false # disable progress display webVideo: null # process the video from URL alpha: false # preserve the Alpha channel for transparent image ffmpeg: ffmpeg_path: '/usr/bin/' intermediate_file_name: 'intermediate.mkv' # Step 1: Frame Extraction # extract all frames from original input into temporary directory extract_frames: '-hwaccel': auto # automatically select hardware acceleration method '-y': true input_options: {} # empty dict, expand if necessary output_options: '-pix_fmt': rgb24 # extracted frames pixel format '-qscale:v': null # output image quality control for JPEG (1-31, 1 being highest) # Step 2: Video Assembly # assemble all frames back into a video with only a video track assemble_video: '-hwaccel': auto # automatically select hardware acceleration method '-y': true input_options: '-f': image2 # force image2 format output_options: '-vcodec': h264_nvenc # video codec '-pix_fmt': 'yuv420p' # overwrite default pixel format '-crf': 17 # H.264 Constant Rate Factor '-b:v': null # target average bitrate '-vf': 'pad=ceil(iw/2)*2:ceil(ih/2)*2' # ensure output is divisible by 2, recommended for libx264 '-tune': 'animation' # encoding tuning film/animation/grain/stillimage/fastdecode/zerolatency/psnr/ssim # Step 3: Streams Migration # migrate audio and subtitle streams from original # video into the upscaled video migrate_streams: '-hwaccel': auto # automatically select hardware acceleration method '-y': true input_options: {} # empty dict, expand if necessary output_options: '-map': - '0:v?' # copy video streams - '1:a?' # copy audio streams - '1:s?' # copy subtitle streams - '1:d?' # copy data streams - '1:t?' # copy fonts '-c': copy # copy codec for all streams # '-vf': 'minterpolate=''fps=60''' # minterpolate frame interpolation '-map_metadata': 0 # copy known metadata tags # '-movflags': 'use_metadata_tags' # copy custom/arbitrary metadata tags '-pix_fmt': null '-metadata': 'comment=Upscaled by Video2X' gifski: gifski_path: '/usr/bin/gifski' # output: null # Destination file to write to # fps: 20 # Animation frames per second (for PNG frames only) [default: 20] fast: false # 3 times faster encoding, but 10% lower quality and bigger file quality: 100 # Lower quality may give smaller file #width: null # Maximum width #height: null # Maximum height (if width is also set) once: false # Do not loop the GIF nosort: false # Use files exactly in the order given, rather than sorted quiet: false # Do not show a progress bar video2x: video2x_cache_directory: /home/neko-san/.cache/video2x # default: %TEMP%\video2x, directory where cache files are stored, will be deleted if preserve_frames is not set to true extracted_frame_format: png # png/jpg intermediate file format used for extracted frames during video processing output_file_name_format_string: "{original_file_name}_upscaled{extension}" # format string to use for generating output file names image_output_extension: .png # image output extension during batch processing video_output_extension: .mkv # video output extension during batch processing preserve_frames: false # if set to true, the cache directory won't be cleaned upon task completion ```

[mirh]

Did you try to run the command by itself?

/usr/bin/ffmpeg -r 29.419795221843003 -hwaccel auto -y -f image2 -i /home/neko-san/.cache/video2x/tmptp_c75mh/extracted_%d.png -vcodec h264_nvenc -pix_fmt yuv420p -crf 17 -vf pad=ceil(iw/2)*2:ceil(ih/2)*2 -tune animation /home/neko-san/.cache/video2x/tmptp_c75mh/intermediate.mkv

There are two weird errors here

Codec AVOption crf (Select the quality for constant quality mode) specified for output file #0 (/home/neko-san/.cache/video2x/tmptp_c75mh/intermediate.mkv) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
  Stream #0:0 -> #0:0 (png (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264_nvenc @ 0x561d9e27d440] [Eval @ 0x7ffc87379a78] Undefined constant or missing '(' in 'animation'
[h264_nvenc @ 0x561d9e27d440] Unable to parse option value "animation"
[h264_nvenc @ 0x561d9e27d440] Error setting option tune to value animation.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

And last but not least, everybody uses rad tools to encode to bink.

[k4yt3x]

Looks like you're using an old version of Video2X on Gentoo, which I would not recommend albeit it may not have anything to do with the error. Consider Docker or Colab.

Follow what @mirh said to see if FFmpeg can work with the format at all. If not, convert to another common format before processing it.