search

Filarius / video2video

Automatic1111 Stable Diffusion WebUI Video Extension
87 stars 14 forks source link

How to get started #2

Open digimbyte opened 1 year ago

digimbyte commented 1 year ago

I'm an idiot with this stuff, I have a video file locally, I clone this repo, then what do I do? does this package require other software? does it require any installation commands, etc?

Filarius commented 1 year ago

Are you on Windows, or Linux? It needs FFmpeg and Python module sk-video I wish i fixed bugs related to "there is no sk-video", so it will be installed at first (re)start. For Win it will install FFmpeg automaticaly, for *Nix you need to install in manually

Place any video file into input of pluging, but i recommend to set path to file (I have UI lagging while uploading a video, and no lags if I set path to local file). Choose FPS, plugin will change FPS of video as you set it, droping frames whats not fit to low FPS. Its good if you want to do a test run. Keep input frames checkbox will make it do not drop any input frames (so you can do like of slow-mo, or speedup). Output video FPS will be set by slider. Setting both checkboxes will make output FPS same as input (but i find there 1 frame can be dropped somewhy), ignoring FPS slider

Latent blending - makes it mix latent spacee between frames while SD doing each step. Scale to 1.0 means frames will be mixed as 50%-to-50%, Scale 0.5 means there 75% of current frame and 25% of previous frame Scaling factor means the scale will be changed each step, at each step it will be multipled by this value. So its like setting this less then 1.0 will make scale be lower at final steps, so there will be less mixing of "small details" (if it can be said so) while harder mixing of "big details" at start.

SethBonser commented 1 year ago

do the files for this need to be copied into the automatic1111 repo and then it will just install and run automatically or do I need to do anything else to get it in the right place?

Filarius commented 1 year ago

do the files for this need to be copied into the automatic1111 repo and then it will just install and run automatically or do I need to do anything else to get it in the right place?

Files to run like ffmpeg and ffprobe,

Video files - can be located anywhere on local PC, or even in internet to download by direct link (just use URL in thr box for file path, not actually tested, but ffmpeg must support this).

digimbyte commented 1 year ago

So I am aware of FFmpeg, and typically you download a dll file and host it on your main drive path. I see comments that its auto installed with windows and a pip command for sk-video do I run the pip command in the local directory or is that globally automatically?

As for this source, do I need to insert it into or overtop the stable diffusion webui in a particular folder?

digimbyte commented 1 year ago

to clarify, I ended cloning and using the extensions "install from url" but there is no video2video tab

Filarius commented 1 year ago

do I run the pip command in the local directory or is that globally automatically?

I expect sk-video will be installed auto at any OS. Maybe at "install from URL" you need to restart Auto1111, i'm not sure.

As for this source, do I need to insert it into or overtop the stable diffusion webui in a particular folder?

There must be folder SD\extensions\video2video\ if you install from WEB UI possibly you can place it manually, but update will not work. At "old times" we put scitps into scripts folder.

to clarify, I ended cloning and using the extensions "install from url" but there is no video2video tab

Not a "tab", but option in drop-down menu at bottom of img2img tab

bfvogel commented 1 year ago

This script is functioning but I am getting odd results. If I leave controlnet disabled then the output does not use any of the source video frames for reference outside of what appears to be Img2Img so there is too much drift. If I enable controlnet to get a bit more pose consistency frame to frame then the ControlNet detectmaps get injected into the video as well. The resulting video it one frame of output then a controlnet detectmap then the next frame / next detectmap and so in. I'm sure I am just missing something.