lllyasviel / stable-diffusion-webui-forge

GNU Affero General Public License v3.0
7.72k stars 739 forks source link

[Bug]: depth_hand_refiner Preprocessing error #566

Open hben35096 opened 6 months ago

hben35096 commented 6 months ago

Checklist

What happened?

PixPin_2024-03-17_00-40-28

Steps to reproduce the problem

  1. In ControlNet, upload the image,
  2. Select "depth_hand_refiner" and click the "Run preprocessor" button

What should have happened?

Generate a depth image with complete pre-processing

What browsers do you use to access the UI ?

Microsoft Edge

Sysinfo

sysinfo-2024-03-16-16-46.json

Console logs

/mnt/workspace/stable-diffusion-webui-forge
Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
Version: f0.0.17v1.8.0rc-latest-276-g29be1da7
Commit hash: 29be1da7cf2b5dccfc70fbdd33eb35c56a31ffb7
Launching Web UI with arguments: --no-download-sd-model --xformers --enable-insecure-extension-access --theme=dark
Total VRAM 22732 MB, total RAM 30179 MB
xformers version: 0.0.23.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A10 : native
VAE dtype: torch.bfloat16
CUDA Stream Activated:  False
[2024-03-17 00:21:05,867] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Using xformers cross attention
ControlNet preprocessor location: /mnt/workspace/stable-diffusion-webui-forge/models/ControlNetPreprocessor
Loading weights [15012c538f] from /mnt/workspace/stable-diffusion-webui-forge/models/Stable-diffusion/sd15/realisticVisionV60B1_v51VAE.safetensors
2024-03-17 00:21:11,416 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://127.0.0.1:7860
model_type EPS
UNet ADM Dimension 0

To create a public link, set `share=True` in `launch()`.
Startup time: 15.7s (prepare environment: 2.9s, import torch: 5.4s, import gradio: 1.7s, setup paths: 0.4s, other imports: 0.8s, load scripts: 1.6s, create ui: 0.5s, gradio launch: 2.4s).
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
To load target model SD1ClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  22503.1240234375
[Memory Management] Model Memory (MB) =  454.2076225280762
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  21024.916400909424
Moving model(s) has taken 0.05 seconds
Model loaded in 12.2s (load weights from disk: 1.0s, forge load real models: 10.7s, calculate empty prompt: 0.5s).
2024-03-17 00:21:43,704 - ControlNet - INFO - Preview Resolution = 512
Automatic Memory Management: 659 Modules in 0.08 seconds.
2024-03-17 00:21:55,667 - ControlNet - INFO - Preview Resolution = 512
img_size [384, 512]
Params passed to Resize transform:
    width:  512
    height:  384
    resize_target:  True
    keep_aspect_ratio:  True
    ensure_multiple_of:  32
    resize_method:  minimal
Automatic Memory Management: 616 Modules in 0.20 seconds.
2024-03-17 00:22:24,613 - ControlNet - INFO - Preview Resolution = 512
WARNING:trimesh:No FCL -- collision checking will not work
set os.environ[OMP_NUM_THREADS] to 4
/opt/conda/lib/python3.10/site-packages/scipy/sparse/_index.py:100: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
  self._set_intXint(row, col, x.flat[0])
=> loading pretrained model /mnt/workspace/stable-diffusion-webui-forge/models/ControlNetPreprocessor/hand_refiner/hr16/ControlNet-HandRefiner-pruned/hrnetv2_w64_imagenet_pretrained.pth
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1710606154.205859     402 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:84) egl_initializedUnable to initialize EGL
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/opt/conda/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/opt/conda/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/opt/conda/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/opt/conda/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/mnt/workspace/stable-diffusion-webui-forge/extensions-builtin/sd_forge_controlnet/lib_controlnet/controlnet_ui/controlnet_ui_group.py", line 847, in run_annotator
    result = preprocessor(
  File "/mnt/workspace/stable-diffusion-webui-forge/extensions-builtin/forge_legacy_preprocessors/scripts/legacy_preprocessors.py", line 103, in __call__
    result, is_image = self.call_function(img=input_image, res=resolution, thr_a=slider_1, thr_b=slider_2, **kwargs)
  File "/mnt/workspace/stable-diffusion-webui-forge/extensions-builtin/forge_legacy_preprocessors/legacy_preprocessors/preprocessor.py", line 854, in run_model
    depth_map, mask, info = self.model(
  File "/opt/conda/lib/python3.10/site-packages/hand_refiner/__init__.py", line 29, in __call__
    depth_map, mask, info = self.pipeline.get_depth(input_image, mask_bbox_padding)
  File "/opt/conda/lib/python3.10/site-packages/hand_refiner/pipeline.py", line 365, in get_depth
    cropped_depthmap, pred_2d_keypoints = self.run_inference(graphormer_input.astype(np.uint8), self._model, self.mano_model, self.mesh_sampler, scale, int(crop_len))
  File "/opt/conda/lib/python3.10/site-packages/hand_refiner/pipeline.py", line 260, in run_inference
    points, index_ray, _ = intersector.intersects_location(rays_o, rays_d, multiple_hits=False)
  File "/opt/conda/lib/python3.10/site-packages/trimesh/ray/ray_triangle.py", line 89, in intersects_location
    locations) = self.intersects_id(ray_origins=ray_origins,
  File "/opt/conda/lib/python3.10/site-packages/trimesh/ray/ray_triangle.py", line 52, in intersects_id
    locations) = ray_triangle_id(triangles=self.mesh.triangles,
  File "/opt/conda/lib/python3.10/site-packages/trimesh/ray/ray_triangle.py", line 171, in ray_triangle_id
    ray_candidates, ray_id = ray_triangle_candidates(
  File "/opt/conda/lib/python3.10/site-packages/trimesh/ray/ray_triangle.py", line 275, in ray_triangle_candidates
    dtype=np.int)
  File "/opt/conda/lib/python3.10/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'inf'?

Additional information

No response

newxhy commented 6 months ago

me too