search

wenquanlu / HandRefiner

MIT License
701 stars 30 forks source link

mac m2: NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook emplo #25

Open i18n-now opened 5 months ago

i18n-now commented 5 months ago
image 
2024-01-28 16:34:25,167 - ControlNet - INFO - ControlNet model control_sd15_inpaint_depth_hand_fp16 [09456e54] loaded.
2024-01-28 16:34:25,196 - ControlNet - INFO - Using preprocessor: depth_hand_refiner
2024-01-28 16:34:25,197 - ControlNet - INFO - preprocessor resolution = 744
set os.environ[OMP_NUM_THREADS] to 4
街照-744-1200.flow
{'state-ext-adetailer-txt2img_0-textarea_1': 'paintings, sketches, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, u
gly, mutation, mangled, gross proportion, out of focus, mutilated hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-ext-adetailer-txt2img-toggled': 'true', 'state-ext-control-net-txt2img_0-model': 'ip-adapt
er-faceid-plusv2_sdxl [187cb962]', 'state-txt2img_seed': '3777285790', 'state-ext-adetailer-txt2img_1-use-separate-noise-multiplier-2nd': 'true', 'state-ext-dynamic-prompts-txt2img-magic-prompt': 'true', 'state-ext-adetailer-txt2img_1-use-separate-vae-2nd': 'true', 'state-ext-adetailer-t
xt2img_0-use-separate-checkpoint': 'true', 'state-ext-control-net-txt2img-tab': 'ControlNet Unit 0 ', 'state-txt2img_sampling': 'DPM++ SDE Karras', 'state-ext-adetailer-txt2img_1-use-separate-cfg-scale-2nd': 'true', 'state-txt2img_batch_count': '1', 'state-ext-adetailer-txt2img_0-use-sep
arate-sampler': 'true', 'state-setting_sd_model_model': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-dynamic-prompts-txt2img-combinatorial-generation': 'false', 'state-ext-adetailer-txt2img-enable-adetailer': 'true', 'state-ext-adetailer-txt2img_1-textarea_2': 'B
est quality, masterpiece, ultra high res, photorealistic, raw photo,\n1gril,look at viewer,\ngirl face, sexy lip,perfect eyes,\nlong hair,high bangs,\nyangmi,\n <lora:beautiful_girl_ver2:0.3>,\n <lora:YangmiV81:0.6>, \n<lora:AIGirlXL_V1:0.3>,\n <lora:RMSDXL_Enhance:1>,\n', 'state-txt2img
_batch_size': '1', 'state-tab': 'txt2img', 'state-txt2img_#script_list': 'None', 'state-ext-control-net-txt2img_0-control-weight': '0.5', 'state-ext-dynamic-prompts-txt2img-toggled': 'false', 'state-setting_sd_model_checkpoints': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]',
 'state-ext-adetailer-txt2img_1-use-separate-checkpoint-2nd': 'true', 'state-ext-adetailer-txt2img_1-use-separate-clip-skip-2nd': 'false', 'state-txt2img_cfg_scale': '1.5', 'state-txt2img_height': '1200', 'state-ext-adetailer-txt2img_1-use-separate-sampler-2nd': 'true', 'state-txt2img_st
eps': '7', 'state-ext-adetailer-txt2img_1-adetailer-steps-2nd': '15', 'state-ext-adetailer-txt2img_0-use-separate-vae': 'true', 'state-ext-adetailer-txt2img_0-inpaint-denoising-strength': '0.53', 'state-ext-control-net-txt2img_0-checkpoints': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', '
state-ext-adetailer-txt2img_1-adetailer-model-2nd': 'hand_yolov8n.pt', 'state-ext-adetailer-txt2img_1-use-separate-steps-2nd': 'true', 'state-setting_sd_vae': 'Automatic', 'state-ext-adetailer-txt2img_1-inpaint-denoising-strength-2nd': '0.5', 'state-txt2img_neg_prompt': 'paintings, sketc
hes, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, ugly, mutation, mangled, gross proportion, out of focus, mutila
ted hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-setting_sd_model_models': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-adetailer-txt2img_0-use-separate-steps': 'true', 'state-
ext-control-net-txt2img_0-preprocessor': 'ip-adapter_face_id_plus', 'state-ext-adetailer-txt2img_1-controlnet-model-2nd': 'control_sd15_inpaint_depth_hand_fp16 [09456e54]', 'state-ext-adetailer-txt2img_0-textarea_0': 'nice hands, perfect hands, girl hands,detailed hand,Hand close-up\n',
'state-ext-adetailer-txt2img_1-adetailer-checkpoint-2nd': 'dreamshaper_8.safetensors', 'state-txt2img_width': '744', 'state-ext-control-net-txt2img_0-models': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', 'state-ext-adetailer-txt2img_1-controlnet-module-2nd': 'depth_midas', 'state-ext-adet
ailer-txt2img_1-textarea_3': 'paintings, sketches, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, ugly, mutation, m
angled, gross proportion, out of focus, mutilated hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-ext-control-net-txt2img_0-checkpoint': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', 'state-ext-adetailer-tx
t2img-tab': '2nd ', 'state-setting_sd_model_checkpoint': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-adetailer-txt2img_1-adetailer-sampler-2nd': 'Restart', 'state-ext-control-net-txt2img-toggled': 'true', 'state-ext-control-net-txt2img_0-enable': 'true', 'state-
txt2img_prompt': 'Best quality, masterpiece, ultra high res, photorealistic, raw photo,\nwave hand, nice hand,arm up, detailed hand,\n full shot , Volume light,\n1gril, sexy pose,\n{glasses|coloured pupil|Watery|black|blue} eyes,\n{dimple|smile|sad|shy|anger|weep} face,\nblack hair,\ngir
l face, sexy lip,perfect eyes,\nlong hair,high bangs,\nyangmi,\n <lora:beautiful_girl_ver2:0.3>,\n <lora:YangmiV81:0.6>, \n<lora:AIGirlXL_V1:0.3>,\n <lora:RMSDXL_Enhance:1>,\nlook at viewer,\n{police uniform|Cycling Jersey and Shorts|Flight Attendant Uniform| Multi Pockets Jumpsuit Mecha
nic Uniform|Doctor‘s White Coat|Tennis Dress|Maid outfit| Golf Polo Shirt and Pants|Construction Worker’s Safety Vest|wedding dress}\n\n', 'state-ext-adetailer-txt2img_1-adetailer-cfg-scale-2nd': '2'}
using_loras = ['beautiful_girl_ver2', 'YangmiV81', 'AIGirlXL_V1', 'RMSDXL_Enhance', 'beautiful_girl_ver2', 'YangmiV81', 'AIGirlXL_V1', 'RMSDXL_Enhance']
/Users/z/git/stable-diffusion-webui/models/Lora/beautiful_girl_ver2.safetensors
Use Memory Optimized SHA256
/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/scipy/sparse/_index.py:102: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
  self._set_intXint(row, col, x.flat[0])
*** Error running process: /Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
    Traceback (most recent call last):
      File "/Users/z/git/stable-diffusion-webui/modules/scripts.py", line 718, in process
        script.process(p, *script_args)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 1138, in process
        self.controlnet_hack(p)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 1123, in controlnet_hack
        self.controlnet_main_entry(p)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 930, in controlnet_main_entry
        detected_map, is_image = self.preprocessor[unit.module](
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 80, in decorated_func
        return cached_func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 64, in cached_func
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 858, in run_model
        self.load_model()
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 844, in load_model
        self.model = MeshGraphormerDetector.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/hand_refiner/__init__.py", line 17, in from_pretrained
        pipeline = MeshGraphormerMediapipe(args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/hand_refiner/pipeline.py", line 76, in __init__
        mesh_sampler = Mesh(device=args.device)
                       ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/mesh_graphormer/modeling/_mano.py", line 124, in __init__
        self._U = [u.to(device) for u in self._U]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/mesh_graphormer/modeling/_mano.py", line 124, in <listcomp>
        self._U = [u.to(device) for u in self._U]
                   ^^^^^^^^^^^^
    NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook emplo
yee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty.memory_format' is only available for these backends: [CPU, MPS, Meta, QuantizedCPU, QuantizedMeta, MkldnnCPU, SparseCPU, SparseMeta, SparseCsrCPU, BackendSelect, Python, FuncTorchD
ynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1,
AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

    CPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterCPU.cpp:31034 [kernel]
    MPS: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterMPS.cpp:22748 [kernel]
    Meta: registered at /dev/null:241 [kernel]
    QuantizedCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterQuantizedCPU.cpp:929 [kernel]
    QuantizedMeta: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterQuantizedMeta.cpp:105 [kernel]
    MkldnnCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterMkldnnCPU.cpp:507 [kernel]
    SparseCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseCPU.cpp:1379 [kernel]
    SparseMeta: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseMeta.cpp:249 [kernel]
    SparseCsrCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseCsrCPU.cpp:1128 [kernel]
    BackendSelect: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterBackendSelect.cpp:726 [kernel]
    Python: registered at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/core/PythonFallbackKernel.cpp:144 [backend fallback]
    FuncTorchDynamicLayerBackMode: registered at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/functorch/DynamicLayer.cpp:491 [backend fallback]
kavinbj commented 5 months ago

same in mac m1pro, maybe not support MPS

misato2099 commented 4 months ago

Same problem here. Can Macbook install the hand refiner directly instead of using WEBUI contrlnet?