search

hollowaykeanho / Upscaler

A consolidation of various compiled open-source AI image/video upscaling product for a working CLI friendly image and video upscaling program.
BSD 3-Clause "New" or "Revised" License
252 stars 20 forks source link

Increase Video Processing from 1 Frame to Multi Frames in Parallel #1

Open hollowaykeanho opened 1 year ago

hollowaykeanho commented 1 year ago

Description

Please provide a short description of what feature you're looking forward to
enhance below. Please include the story behind your idea as well to give a
better visualization of your idea.

Basically to speed things up.

Expected Behavior

Please specify the expected behavior of your requested enhancement. Some great
and helpful pointers are your expected interface (e.g. command patterns,
simple sketches of the user interface, etc).

Upscaler can work a lot faster via work.

Current Behavior

Please specify the current behavior (e.g. workaround, blockage, etc).

Upscaler only work on 1 frame at a time.

Attachment

Please drag and drop the necessary data files (e.g. screenshot, logs, etc)
below.
hollowaykeanho commented 1 year ago

Help needed as I do not have a better graphics card. Limited to only NVIDIA GeForce MX150 from a laptop to test its working capability on consumer products.

hollowaykeanho commented 1 year ago

Not possible with POSIX and Powershell. Neither of them provides some kind of sync mechanism like mutex lock or channels. Closing this issue as won't fix.

hollowaykeanho commented 1 year ago

Re-open as now we have a possibility to process frame upscaling concurrently using scoring system.

hollowaykeanho commented 1 year ago

@Joly0, @JeanShuralyov , can I acquire --parallel argument name for experimenting this feature ya?

like --parallel 2 for 2 images simultenously.

Joly0 commented 1 year ago

Sounds good for me, i´d have to adjust the powershell side of this when you are done

hollowaykeanho commented 1 year ago

TQ & noted. I'll begin the enhancement work after clearing some works at hand.

JeanShuralyov commented 1 year ago

@hollowaykeanho , fine by me. Will you be using my fork repo for this enhancement?

hollowaykeanho commented 1 year ago

yes. Please keep it.

hollowaykeanho commented 1 year ago

@Joly0 , I might need help with your graphics card. =x

Appearently, I completed my prototyping codes but same like my previous attempt, I blew my vRAM when parallel is set to 2. (It demands 2400MB I believe).

2023-07-13-19-50-28

I will complete the commit and push to Jean's forked repo for easier referencing.

Joly0 commented 1 year ago

Sure, how can i help?

hollowaykeanho commented 1 year ago

Okay. I just pushed in parallel processing capability for init/unix.sh side of things. Diff: https://github.com/hollowaykeanho/Upscaler/commit/aa4fabb18a8256159ccb5555f787f06546c33536

You can try it out with --parallel 2 (I can't get through due to the hardware limitation). Once translated to the PowerShell side, this issue can be marked completed. =)

Heads up: the unix.sh hits the 1000 lines warning limit already, anything beyond will need a proper progrmaming language which also means: Issue #34.

Growing beyond 1000 lines for shell scripts are nightmare to maintain in near future.

JeanShuralyov commented 1 year ago

@hollowaykeanho , there is no way my Iris can operate --parallel 2 if your Nvidia failed. Passed. Otherwise, great job.

hollowaykeanho commented 1 year ago

noted. In that case I won't increase its parallelism for benchmark testing then since we do not have proper hardware to test it iteratively.

hollowaykeanho commented 1 year ago

@Joly0 , is it possible to translate the current outcome ya? It will be the last one since I'll drilling into Tencent NCNN framework for the next step.

Joly0 commented 1 year ago

Hey, yes, i am on it getting this done in powershell, but i dont have that much time currently, so i cant tell, when i am finished. I will try my best to get this done asap.

hollowaykeanho commented 1 year ago

Haha.. no problem. It's open-source so no deadline. Take your time.

In that case I will begin the work on the source codes now. Let me know when you're ready for release.

hollowaykeanho commented 1 year ago

Just in case: please note that you're not required to create directory for each frames that I used.

I only did that because mkdir can behave safely in Linux side for concurrency application. You might need to find out is PowerShell has a similar command that has a safe concurrent mechanism.

Joly0 commented 1 year ago

Btw, in powershell this is a hell of a ride, but i am getting quite close i guess, but i have to set powershell 7 as a requirement for this to work. Powershell 5 has a few techniques to make parallel execution working (jobs, runspaces, workflows), but none of them are easy to work with. I could try to get it working with jobs or workflows, but its such a pain to work with functions and variables outside of a job or workflow scope. Powershell 7 has the "foreach-object -parallel " function, which works atleast better. Still having issues with variables and functions, but its atleast not too difficult to work with.

So if its ok, i will get this working with powershell 7, otherwise if you want, i´ll adjust it to powershell 5

But just so you see it, its working image (running the benchmark here with --parallel 2)

I´ll let this run through and will try with some other files aswell and i have to get the control file working again, because it is borked now XD image

Other than that, it looks like it is working

hollowaykeanho commented 1 year ago

Powershell 5 has a few techniques to make parallel execution working (jobs, runspaces, workflows), but none of them are easy to work with.

Hahaha.. completely agree with you. I'm currently working on a PowerShell + POSIX compliant Shell CI infra at https://github.com/ChewKeanHo/AutomataCI/tree/experimental. Currently working on the Docker image packaging job development.

Once I completed the AutomataCI to a stable release, I will port it here and there will be various released packages (for sure having .deb, .rpm, .flatpak, .tar.xz, and .zip.

You can make use of my newly developed shell-based libraries here (e.g. you might be interested in io): https://github.com/ChewKeanHo/AutomataCI/tree/experimental/automataCI/services

(running the benchmark here with --parallel 2)

Good luck! I couldn't run here.

I´ll let this run through and will try with some other files aswell and i have to get the control file working again, because it is borked now XD

Haha. Yeap. Some changes done because the execution is different now.

Joly0 commented 7 months ago

Btw, just to give a quick update. I havent had any time in the past few months to look into this at all, which i am quite sorry for. But i think i´ll might have some time now to get this a all fixed, finish it up and push it out, so it can be merged :D

corygalyna commented 7 months ago

@Joly0 , I had cleaned up the project dashboard into a road-map kanban style already. This should further the repository development with a more directional approach.

Yawn. Enjoy. Oh by the way, @hollowaykeanho , did something mad in this project: https://github.com/ChewKeanHo/AutomataCI/tree/main/automataCI/services

I believe controlled multi-threading is available in there io/sync.*