ShihaoZhaoZSH / Uni-ControlNet

[NeurIPS 2023] Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models
MIT License
574 stars 41 forks source link

Image inference process unresponsive #19

Closed Sonettovo closed 9 months ago

Sonettovo commented 9 months ago

Thank you for your great work! However, I couldn't follow your tutorial to generate the image. After I downloaded your model file, I tried multiple combinations but nothing worked.(I implemented it strictly according to your instructionsimage the detailed information as follows:

logging improved. Enabled sliced_attention. 2023-12-07 14:35:02.777289: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-12-07 14:35:05.978474: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT cuda cuda /home/tdt/.conda/envs/ldm/lib/python3.8/site-packages/timm/models/_factory.py:117: UserWarning: Mapping deprecated model name vit_base_resnet50_384 to current vit_base_r50_s16_384.orig_in21k_ft_in1k. model = create_fn( Use Checkpoint: False Checkpoint Number: [0, 0, 0, 0] Use global window for all blocks in stage3 load checkpoint from local path: ./annotator/ckpts/upernet_global_small.pth UniControlNet: Running in eps-prediction mode Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. DiffusionWrapper has 859.52 M params. making attention of type 'vanilla-xformers' with 512 in_channels building MemoryEfficientAttnBlock with 512 in_channels... Working with z of shape (1, 4, 32, 32) = 4096 dimensions. making attention of type 'vanilla-xformers' with 512 in_channels building MemoryEfficientAttnBlock with 512 in_channels... Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads. Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads. Loaded model config from [./configs/uni_v15.yaml] Loaded state_dict from [./ckpt/uni.ckpt] Running on local URL: http://0.0.0.0:7860

Sonettovo commented 9 months ago

after Keyboard: ^CKeyboard interruption in main thread... closing server. Task exception was never retrieved future: <Task finished name='nj1hbuiwno_0' coro=<Queue.process_events() done, defined at /home/tdt/.conda/envs/ldm/lib/python3.8/site-packages/gradio/queueing.py:324> exception=AssertionError('No event data')> Traceback (most recent call last): File "/home/tdt/.conda/envs/ldm/lib/python3.8/site-packages/gradio/queueing.py", line 338, in process_events response = await self.call_prediction(awake_events, batch) File "/home/tdt/.conda/envs/ldm/lib/python3.8/site-packages/gradio/queueing.py", line 298, in call_prediction assert data is not None, "No event data" AssertionError: No event data

Sonettovo commented 9 months ago

Sorry,I solve this problem after I upgrade the gradio version: pip install gradio==3.40.1 solved it for me

accoring to this issue: https://github.com/lllyasviel/ControlNet/issues/492#issuecomment-1675073320