SuperMarcus
commented
3 years ago 3x or higher upscaling ratios aren't part of our models, so it's a little harder to get them to work. You might need to train your own models (unless somebody already trained them). Currently, the app only does one pass for each frame. You might be able to run the model twice to do 4x scaling. Everything related to the upscaling process should be in ConvertingAsset.swift.
0.4fps does seem a little slow. I'm also using a 5700xt, and it usually sits around 5-10fps from 480p to 720p. But since the app is running the model for every frame, you shouldn't really expect the performance to be real-time. There are projects that use smarter algorithms to improve the performance of video upscaling (e.g., Dandere2x). There are also other non-NN/purely CV-based algorithms (namely Anime4K) that offer way better performance than Waifu2x frame-by-frame upscaling. These are all things that we can potentially integrate into this application.
How about 3x or 4x? I got the app rebuilt for Catalina (the release was not launching).
Started poking around some of the code. Not entirely comfortable with everything yet. It seems things might be a bit redundant, but again, I'm not entirely sure yet. I was hoping adding a 3x or 4x upscale would be relatively trivial, but it appears I have to build a whole new set of classes for that. Not sure why there isn't one class that takes in the set of params for the engine.
Anyway, any chance we could get a 3x or 4x? I want to upscale a 480p source into 1080p. 2x falls short at 960. 3x technically is over kill at 1440, but it could be scaled back down after. Even more ideal would be able to pick the output size, but I'm not sure if that's possible with waifu2x.
Anyway, thanks for making this! Takes a lot of the tedium out of the process. I'm on an AMD 5700XT, and this is running at .4fps. Any way to speed this up?