search

AbdBarho / stable-diffusion-webui-docker

Easy Docker setup for Stable Diffusion with user-friendly UI
Other
6.78k stars 1.13k forks source link

[Request] Deforum plug-in for Auto #148

Closed ProducerMatt closed 2 years ago

ProducerMatt commented 2 years ago

Deforum's scripts provide video-generation functionality with camera movements. They can be installed as a plug-in with this repo: https://github.com/deforum-art/deforum-for-automatic1111-webui

I'm trying to get it working in your repo. With these changes I can get the Deforum plug-in loaded in the UI, but running it produces the Python error "No module named 'py3d_tools'". I get conflicting info on whether the package for py3d_tools is open3d-python or pytorch3d, though I tried both.

AbdBarho commented 2 years ago

@ProducerMatt I am not sure how I can help you, I haven't used the script myself so it is hard to tell.

That being said, your changes seem correct, the problem is something else.

StableInquest commented 2 years ago

@ProducerMatt Did you solve this. I'm having the same issue. Python error "No module named 'py3d_tools'"

ProducerMatt commented 2 years ago

@StableInquest Nope. I tried with the new version and I still get the same problem. https://github.com/AbdBarho/stable-diffusion-webui-docker/compare/master...ProducerMatt:stable-diffusion-webui-docker:auto-ext

The only bread crumb I've found, someone had this problem because their deforum folder was not in (your auto repo)/extensions/deforum. Apparently the library locations are hard coded.

AbdBarho commented 2 years ago

here is how you setup Deforum:

  1. checkout the branch #176 auto-extensions
  2. mkdir -p data/config/auto/extensions
  3. git clone https://github.com/deforum-art/deforum-for-automatic1111-webui/ data/config/auto/extensions/deforum
  4. create a file data/config/auto/startup.sh and put the following in it: 
    
set -e

pip install -q -r extensions/deforum/requirements.txt

PY3D_PATH="${PWD}/extensions/deforum/py3d" ADA_PATH="${PWD}/extensions/deforum/AdaBins" MIDAS_PATH="${PWD}/extensions/deforum/midas"

if [ ! -d $PY3D_PATH ]; then git clone https://github.com/MSFTserver/pytorch3d-lite.git $PY3D_PATH fi if [ ! -d $ADA_PATH ]; then git clone https://github.com/shariqfarooq123/AdaBins.git $ADA_PATH fi if [ ! -d $MIDAS_PATH ]; then git clone https://github.com/isl-org/MiDaS.git $MIDAS_PATH fi

apt-get install wget -yq export PYTHONPATH="$PY3D_PATH:$ADA_PATH:$MIDAS_PATH"


6. run `docker compose --profile auto up --build`
7. you should see a new tab called deform, click generate, it should work but would probably take hours

This is only 2D, I haven't tried 3D.

Example, default prompt, 384px 17steps 30 frames:
![download](https://user-images.githubusercontent.com/24505302/197884720-f46fc644-d9a3-4c42-be88-352780ba0723.gif)

Getting this plugin to work was extremely unpleasant, the setup process is not well documented (or I have missed something)

this branch is still a work in progress, so  the setup might change, however, it is obvious to me that maintaining the extensions would be a nightmare, therefore I would leave it up to the user.