Moebits / waifu2x

waifu2x image upscaling in Node.js
MIT License
70 stars 11 forks source link

Parallel video/gif frames upscale #9

Closed Specy closed 3 years ago

Specy commented 3 years ago

Upscaling videos is slow mainly becasue the frames are upscaled one after the other, this uses only around 10/20% of the system resources, the upscaleGif and upscaleVideo functions could "chunk" the frames and upscale them 2 at a time or 3 (ideally have this as an option with default bring 1 or 2).

I'm doing something similar in my program to upscale more than one file at a time and it runs fine even with 5 files, i'm going to share the code I used, in case it could be of any help, i'm using semophores to have the best possible outcome but it's not necessary and can be just done with arrays and for each

const Semaphore = require("semaphore")

function withSemaphore(sem, f) {
    return new Promise((resolve) => {
        sem.take(() => f().then(sem.leave).then(resolve))
    })
}

let maxFrames = 2
async function upscaleVideo(frames){
  let sem = new Semaphore(maxFrames)
  await Promise.all(frames.map(file => {
      return withSemaphore(sem, () => upscaleImage(file))
  }))
}

async function upscaleImage(img){
    . . . 
}

REMEMBER, this does not wait for both frames to finish, but it makes MAX 2 frames upscales at a time (which is even better in theory) There's also a function to clear the semaphore which can be used to stop upscaling

Specy commented 3 years ago

Also I'm making a video upscale from 480p to 1080p at 30 FPS , clip lasts 40 seconds. Total frames 1200, I'm at 250 frames after 35 mins lol. Adding this feature could save upscale times by A LOT

EDIT: 3 hours in and im at 1050 frames

EDIT 2: 4 hours in, it finished but errored out at the encoding with error:

Error: ffmpeg exited with code 1: Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[1] Conversion failed!
[1]
[1]     at ChildProcess.<anonymous> (C:\Users\dione\Desktop\Progetti\electron\scapix\node_modules\fluent-ffmpeg\lib\processor.js:182:22)
[1]     at ChildProcess.emit (events.js:315:20)
[1]     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)

EDIT 4: Turns out ffmpeg can only encode even numbers... the images were 1921x1080 so it errored out...

Moebits commented 3 years ago

You can increase the parallel frames with the option parallelFrames (default is 1). I don't recommend setting this too high because your computer will freeze if it runs out of CPU/memory, that's why I didn't make it parallel in the first place. Make sure you are on v0.3.9.