FFMPEG error for width / height that is not even

[Specy]

whenever a frame after upscale is even / odd, it will cause an error with encoding the video, i've found this fix on SO

-vf "crop=trunc(iw/2)*2:trunc(ih/2)*2"

Also, the parallel upscale seems to be working fine, i have to keep testing a bit tho.

I still can't get max use of the system resources... with 5 parallel frames i use 30% cpu and 20% gpu while i have vscode, chrome, discord and the live server open... I wonder if im missing some settings to use hardware acceleration with GPU

[Specy]

Also, i've seen that videos are around 10/20% longer than the original one,

Original: 30s

https://user-images.githubusercontent.com/59029985/106637995-a814f900-6583-11eb-8467-ab14ffb5eb7c.mp4

Upscaled: (its 1.1 scale, ive made it only to test audio, takes around 5/10 mins to finish upscaling this ), im using ffprobe with options:

            let videoOptions = {
                quality: 0,
                noise: noise,
                scale: 1.1,
                ffmpegPath: "path/to/ffmpeg.exe",
                ffprobePath: "path/to/ffprobe.exe",
                parallelFrames: 8, //yes i managed to push 8 parallel frames for this video, issue also happens when value is 1
            }

length: 35s

https://user-images.githubusercontent.com/59029985/106638037-b236f780-6583-11eb-9aa0-bbf677410cd7.mp4

This also causes audio to be out of sync from the video, do you know what could be causing this?

[Moebits]

This also causes audio to be out of sync from the video, do you know what could be causing this?

Idk, but the framerate of that video is really weird at 29.74. But we have a whole number of images, so you end up with a couple more images then the original. A possible fix is to calculate how much longer the new video is, and then re-encode a sped up version of it.

[Specy]

I initially discovered this while upscaling a video which was exactly 30 FPS so it doesn't really seem like the FPS fault, the "fix" wouldn't really guarantee precision in videos as ideally you should have the same output as the input only with the frames changed.

I don't know ffmpeg a lot so I can't really say what could be causing It but I'm going to search around for what's causing it.

1:1 comparison

https://user-images.githubusercontent.com/59029985/106663352-e4585180-65a3-11eb-9fc2-46596cc3a40b.mp4

Both videos have the same frame rate, the virst video has 893 frames, upscaled one has 1062

[Moebits]

I found a solution here which is to use the fps filter instead of the "-r" option to extract the frames. I'm not sure why this is exactly, if you think it's a bug then bring it up to ffmpeg. I also added your cropping filter to the options -> v0.4.1.

[Specy]

Thanks, I'll look at it tomorrow and update you. Also fun fact, what took me 3 hours to upscale (1 minute video at 70 FPS , 720p to 1080p at high denoise) on my pc, took only 25 mins on a rtx3080. At that rate it's pretty usable

[Moebits]

Actually that solution doesn't work, I fixed it with my original idea of re-encoding the video to match the correct speed-> v0.4.2. For example, a 30 seconds video with speed: 1 will have the output corrected to exactly 30 seconds, and with speed: 2 it will be corrected to be exactly 15 seconds long.

[Specy]

Good, I'll let you know

[Specy]

Nope i still have issues when upscaling a video that has final resolution with an odd number (211x158)

Error: ffmpeg exited with code 1: Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height       
[1] Conversion failed!
[1]
[1]     at ChildProcess.<anonymous> (C:\Users\dione\Desktop\Progetti\electron\scapix\node_modules\fluent-ffmpeg\lib\processor.js:182:22)
[1]     at ChildProcess.emit (events.js:315:20)
[1]     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)

maybe it has to be added here? https://github.com/Tenpi/waifu2x/blob/278a75eff6a6c5e609b180cd794b0b2fbb3807c3/waifu2x.ts#L393

Audio is still a bit behind :/ (video still is a tiny bit longer)

https://user-images.githubusercontent.com/59029985/106734747-abf05c00-6613-11eb-86d7-b3d3a7b88fda.mp4

[Moebits]

maybe it has to be added here?

Yeah, thanks. -> v0.4.3

Audio is still a bit behind :/ (video still is a tiny bit longer)

I don't currently have a better solution for this problem. Videos can have an uneven fps or even variable fps, which doesn't carry over well when converting it to an image sequence.

[Specy]

I saw this on SU https://superuser.com/a/1134688 , might help. Even tho I saw you are already using some of those things. Also look at the "first frame should be ignored"

[Specy]

with v0.4.3 upscaling a video results in a completely black video, since the only thing changed is the crop filter i suggested to add, i guess there was some problem with the implementation?

https://github.com/Tenpi/waifu2x/blob/83e2bae5fcaf4ba3854ac1851b493d92e400ec88/waifu2x.ts#L397

Result of the same video sent in other comments:

https://user-images.githubusercontent.com/59029985/106958348-878e9f80-6739-11eb-8546-a1daa4297f76.mp4

[Moebits]

The problem was that I was using options.speed in the final conversion, but if you didn't set it it would be undefined. So then ffmpeg will get NaN as the factor, and I honestly don't know why that doesn't error, but ok. -> v0.4.4.