Possible to run?

[drphero]

It looks like you have achieved some very nice results and I wanted to test this out. I've spent the past two hours trying to get this to run. There are several dependency version issues, mostly caused by installing torch via the requirements file I think. Removing those and executing this first should help:

pip install torch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 --index-url https://download.pytorch.org/whl/cu117

I'm not sure if this has any effect, but the recommended way to install mmcv-full is:

pip install -U openmim
mim install mmcv-full==1.7.0

There is also a missing import sys in pipline_StableDiffusion_ConsistentID.py, so I'm not sure how you actually ran this at all without that.

It's unclear what exactly needs to go into base_model_path = "" and consistentID_path = "". I assumed the base model was an SD1.5 model, but it refused to load the diffusers version of RealisticVision v5.1.

It seems to be necessary to copy the model.py and resnet.py from https://github.com/zllrunning/face-parsing.PyTorch, which info about the existence of only exists in a pull request (the necessary BiSNet model as well).

The necessity of such an old version of cuda (11.7) adds some more trouble.

I find that it is always a good idea to start from a completely fresh environment and see if following the instructions in your README leads to success or not.

[JackAILab]

It looks like you have achieved some very nice results and I wanted to test this out. I've spent the past two hours trying to get this to run. There are several dependency version issues, mostly caused by installing torch via the requirements file I think. Removing those and executing this first should help:

pip install torch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 --index-url https://download.pytorch.org/whl/cu117

I'm not sure if this has any effect, but the recommended way to install mmcv-full is:

pip install -U openmim
mim install mmcv-full==1.7.0

There is also a missing import sys in pipline_StableDiffusion_ConsistentID.py, so I'm not sure how you actually ran this at all without that.

It's unclear what exactly needs to go into base_model_path = "" and consistentID_path = "". I assumed the base model was an SD1.5 model, but it refused to load the diffusers version of RealisticVision v5.1.

It seems to be necessary to copy the model.py and resnet.py from https://github.com/zllrunning/face-parsing.PyTorch, which info about the existence of only exists in a pull request (the necessary BiSNet model as well).

The necessity of such an old version of cuda (11.7) adds some more trouble.

I find that it is always a good idea to start from a completely fresh environment and see if following the instructions in your README leads to success or not.

Thank you for your suggestion.

• The base infer model we use is Realistic_Vision_V6.0_B1_noVAE, which can be downloaded at SG161222/Realistic_Vision_V6.0_B1_noVAE. consistentID_path is the weight of our model and can be downloaded at Google Drive or Baidu Netdisk.

• In fact, there are some unnecessary dependencies in the environment. In theory, you only need to ensure that pytorch ≥ 2.0.0. Other dependencies can be matched according to the server environment.

• The BiSeNet module is available directly from face-parsing.PyTorch. We are starting from scratch to build a new environment and submit a separate implementation of the BiSeNet function to help with better reproduction. Thank you for your patience~

[drphero]

The BiSeNet module is available directly from face-parsing.PyTorch.

Does that mean I should copy all of the files from that repo to the ConsistentID directory?

[drphero]

I see that it had already been added in the latest update. There are some new problems however. Looking at lines 20, 24, and 30 in pipline_StableDiffusion_ConsistentID.py, it seems something accidentally got added that breaks it.

After removing those, now I'm getting OSError: Could not find the necessarysafetensorsweights in {'safety_checker/pytorch_model.bin', 'vae/diffusion_pytorch_model.bin', 'text_encoder/pytorch_model.bin', 'unet/diffusion_pytorch_model.bin'} (variant=fp16) despite SG161222/Realistic_Vision_V6.0_B1_noVAE already having been downloaded manually into my HF cache directory.

It's also weird that I had to manually download it, since before the update app.py automatically downloaded the model that I put into base_model_path.