machineminded
commented
5 months ago I will need to make a separate configuration script for unix-like environments. Shouldn't be too much work. I am not familiar with Google Colab but can make an effort.
Open Rondalter opened 5 months ago
machineminded
commented
5 months ago I will need to make a separate configuration script for unix-like environments. Shouldn't be too much work. I am not familiar with Google Colab but can make an effort.
machineminded
commented
5 months ago @Rondalter I did go in and update the notebook. I am trying to use it but I don't think it will work because it is saying I have zero compute resources available. I'm not familiar with how Colab works but if you could test it out that would be great.
Rondalter
commented
5 months ago equirement already satisfied: pygit2==1.12.2 in /usr/local/lib/python3.10/dist-packages (1.12.2)
Requirement already satisfied: cffi>=1.9.1 in /usr/local/lib/python3.10/dist-packages (from pygit2==1.12.2) (1.16.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.9.1->pygit2==1.12.2) (2.21)
/content/Fooocus-inswapper
[System ARGV] ['launch.py', '--share']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.1.864
Version mismatch for tb-nightly: Installed version 2.16.0a20240202 does not meet requirement tb-nightly
Installing requirements
Total VRAM 15102 MB, total RAM 12979 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla T4 : native
VAE dtype: torch.float32
Using pytorch cross attention
Exception in thread Thread-2 (worker):
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/content/Fooocus-inswapper/modules/async_worker.py", line 25, in worker
import modules.default_pipeline as pipeline
File "/content/Fooocus-inswapper/modules/default_pipeline.py", line 1, in
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run gradio deploy from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Rondalter
commented
5 months ago it stays at that point. I have once managed to generate a basic image. But if you try any option, an error appears.
machineminded
commented
5 months ago On the free tier of Google Colab, how much VRAM is allocated? I think it would need at least 16GB, but even that is probably not enough for use with Fooocus. Maybe 18GB or so. On my local with a 3090, Fooocus is using just over 16GB during generation.
machineminded
commented
5 months ago When I run it, I am only allocated 15GB of VRAM on the free tier. If you are using the free T4 then this could be the issue.
machineminded
commented
5 months ago I should have asked if this was for inswapper or PhotoMaker.
zachysaur
commented
5 months ago Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] Fooocus version: 2.1.864 Version mismatch for tb-nightly: Installed version 2.16.0a20240202 does not meet requirement tb-nightly Installing requirements Total VRAM 15102 MB, total RAM 12979 MB Set vram state to: NORMAL_VRAM Always offload VRAM Device: cuda:0 Tesla T4 : native VAE dtype: torch.float32 Using pytorch cross attention Exception in thread Thread-2 (worker): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/content/Fooocus-inswapper/modules/async_worker.py", line 25, in worker import modules.default_pipeline as pipeline File "/content/Fooocus-inswapper/modules/default_pipeline.py", line 1, in import modules.core as core File "/content/Fooocus-inswapper/modules/core.py", line 1, in from modules.patch import patch_all File "/content/Fooocus-inswapper/modules/patch.py", line 11, in import ldm_patched.modules.sd1_clip File "/content/Fooocus-inswapper/ldm_patched/modules/sd1_clip.py", line 3, in from transformers import CLIPTokenizer ImportError: cannot import name 'CLIPTokenizer' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/init.py) 2024-02-02 18:25:31.015317: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-02-02 18:25:31.015376: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-02-02 18:25:31.021640: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-02-02 18:25:32.883777: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Refiner unloaded. model_type EPS UNet ADM Dimension 2816 Using pytorch attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using pytorch attention in VAE extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'} left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids']) Base model loaded: /content/Fooocus-inswapper/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/content/Fooocus-inswapper/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors]. Loaded LoRA [/content/Fooocus-inswapper/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/content/Fooocus-inswapper/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1. Fooocus V2 Expansion: Vocab with 642 words. /usr/local/lib/python3.10/dist-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() Fooocus Expansion engine loaded for cuda:0, use_fp16 = True. Requested to load SDXLClipModel Requested to load GPT2LMHeadModel Loading 2 new models [Fooocus Model Management] Moving model(s) has taken 0.60 seconds Controls: [textbox, textbox, checkboxgroup, radio, radio, slider, textbox, slider, slider, dropdown, dropdown, slider, dropdown, slider, dropdown, slider, dropdown, slider, dropdown, slider, dropdown, slider, checkbox, textbox, radio, image, checkboxgroup, image, textbox, image, image, slider, slider, radio, image, slider, slider, radio, image, slider, slider, radio, image, slider, slider, radio, checkbox, image, number, checkbox, image, image, image, image] Running on local URL: http://127.0.0.1:7865/ Running on public URL: https://9e35fd3f3edf6da406.gradio.live/
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run gradio deploy from Terminal to deploy to Spaces (https://huggingface.co/spaces)
EntonyHop
commented
5 months ago the same issue, infinite generation of even a normal image. The style button doesn't load either. The fooocus itself doesn't work, I think it's the problem.
machineminded
commented
5 months ago I am still working thru this. It will take some time but hopefully soon.
EntonyHop
commented
5 months ago Let us know when you fix the problem, all work has stopped. Thx
machineminded
commented
5 months ago @EntonyHop Understood - I will try to get this working.
machineminded
commented
5 months ago I am am able to see normal Fooocus features working now in another branch. I found the issue with the inswapper integration on linux and will resolve that next.
machineminded
commented
4 months ago This has been difficult for me to test. But I am making progress. We need a better way to manage the queue:
https://github.com/lllyasviel/Fooocus/issues/2271
Additionally, I believe they have some code in place that detects if you are running Stable Diffusion on the free tier, so the VM/container gets shut down. I don't have a paid subscription otherwise this would be easier for me to test. Will consider paying in order to get this issue closed up.
zachysaur
commented
4 months ago try to kaggle
machineminded
commented
4 months ago try to kaggle
Thanks for the suggestion. I'll check it out!
mashb1t
commented
4 months ago This has been difficult for me to test. But I am making progress. We need a better way to manage the queue:
Additionally, I believe they have some code in place that detects if you are running Stable Diffusion on the free tier, so the VM/container gets shut down. I don't have a paid subscription otherwise this would be easier for me to test. Will consider paying in order to get this issue closed up.
no additional code, but args for better resource allocation: https://github.com/lllyasviel/Fooocus/pull/1710
dmitryalexander
commented
1 month ago does in-swapper work for colab now?
mashb1t
commented
1 month ago @dmitryalexander ~Fooocus (this repository) does not have an inswapper implementation. Please check out https://github.com/machineminded/Fooocus-inswapper maintained by @machineminded.~
omg, nvm, wrong repository. I'm so sorry 😬
dmitryalexander
commented
1 month ago @dmitryalexander ~Fooocus (this repository) does not have an inswapper implementation. Please check out https://github.com/machineminded/Fooocus-inswapper maintained by @machineminded.~
omg, nvm, wrong repository. I'm so sorry 😬
yeah i got the inswapper working, but i couldn't get the instantID or the photo maker to work
machineminded
commented
1 month ago @dmitryalexander Is it just not working in colab? What is the error? I don't have a good way to support colab because I'd have to pay for it to use SD.
dmitryalexander
commented
1 month ago it's only 10$ a month for colab. but yeah, the instant id, it just doesn't activate... it's like it's invisible and the photomaker, it says something like trigger word activated more than once... and it doesn't work from there, i guess that problem happens on some CUDA cards. but apparently the main branch fixed that issue, but i wasn't able to find out how.
machineminded
commented
1 month ago Can you show me the actual log output or a screenshot if you get a chance?
dmitryalexander
commented
1 month ago Traceback (most recent call last): File "/content/Fooocus-inswapper/modules/async_worker.py", line 1047, in worker handler(task) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "/content/Fooocus-inswapper/modules/async_worker.py", line 915, in handler imgs = generate_photomaker(photomaker_prompt, photomaker_source_images, photomaker_negative_prompt, steps, task['task_seed'], width, height, guidance_scale, loras, sampler_name, scheduler_name, async_task) File "/content/Fooocus-inswapper/modules/pm.py", line 119, in generate_photomaker images = pipe( File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/content/Fooocus-inswapper/photomaker/pipeline.py", line 344, in call ) = self.encode_prompt_with_trigger_word( File "/content/Fooocus-inswapper/photomaker/pipeline.py", line 176, in encode_prompt_with_trigger_word raise ValueError( ValueError: PhotoMaker currently does not support multiple trigger words in a single prompt. Trigger word: img, Prompt: completely normal prompt nothing to see here. Total time: 0.47 seconds
dmitryalexander
commented
1 month ago apparently the problem was that there's some kind of space before the trigger word... or something like that?
machineminded
commented
1 month ago Oh, yeah. This is how PhotoMaker works: you need to put "img" in the prompt after "man" or "woman". So your prompt with PhotoMaker enabled might be something like:
"woman img, smile, long hair, standing in a library, incredible lighting"
Check out some of the documentation here:
https://github.com/TencentARC/PhotoMaker?tab=readme-ov-file#use-like-diffusers
"img" is the default trigger word.
dmitryalexander
commented
1 month ago interesting....i'll try it out :thanks:
I am running it in colab, and since the modification that added the photomaker it does not work. You click on generate and it stays blocked.