light-and-ray / sd-webui-lama-cleaner-masked-content

Use lama cleaner before inpainting inside stable-diffusion-webui
85 stars 2 forks source link

object has no attribute 'cpu' #8

Closed RomanS2N closed 3 months ago

RomanS2N commented 4 months ago

Today I updated sd-webui-lama-cleaner-masked-content and I got next an error:

*** Error running before_process: C:\SD\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py
    Traceback (most recent call last):
      File "C:\SD\modules\scripts.py", line 776, in before_process
        script.before_process(p, *script_args)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\scripts\lama_cleaner_masked_content_sctipt.py", line 38, in before_process
        p.init_images[0] = lamaInpaint(p.init_images[0], p.image_mask,
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 82, in lamaInpaint
        tmpImage = processModel(imageRes, maskRes, model)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\inpaint.py", line 42, in processModel
        return lama_model(image, mask)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\model.py", line 41, in __call__
        return processModel(self.model, image, mask)
      File "C:\SD\extensions\sd-webui-lama-cleaner-masked-content\lama_cleaner_masked_content\model.py", line 22, in processModel
        model.cpu()
    AttributeError: 'MaskedImageModelDescriptor' object has no attribute 'cpu'

---
100%|██████████████████████████████████████████████████████████████████████████████████| 18/18 [00:04<00:00,  3.61it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 18/18 [00:06<00:00,  2.64it/s]

Can you fix this issue? I used an old version that I installed 4 months ago and it worked!

RomanS2N commented 4 months ago

and else:

lama inpainted restored from cache
  0%|                                                                                           | 0/16 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(xem86a74rauuiy6)', 2, '', '', [], <PIL.Image.Image image mode=RGBA size=1920x1080 at 0x1BF7FF2D7E0>, None, {'image': <PIL.Image.Image image mode=RGBA size=1920x1080 at 0x1BF7FF2D7B0>, 'mask': <PIL.Image.Image image mode=RGB size=1920x1080 at 0x1BF7FF2D810>}, None, None, None, None, 20, 'DPM++ 2M Karras', 4, 0, 4, 1, 1, 7, 1.5, 0.75, 0.0, 512, 512, 1, 0, 1, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x000001BF7FF2CD90>, 0, False, 1, 0.5, 4, 0, 0.5, 2, False, '', 0.8, -1, False, -1, 0, 0, 0, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 4, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 'DemoFusion', False, 128, 64, 4, 2, False, 10, 1, 1, 64, False, True, 3, 1, 1, True, 0.85, 0.6, 4, False, False, 1536, 96, True, True, True, False, ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=True, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, union_control_type=<ControlNetUnionControlType.UNKNOWN: 'Unknown'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[], batch_keyframe_idx=None), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=True, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, union_control_type=<ControlNetUnionControlType.UNKNOWN: 'Unknown'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[], batch_keyframe_idx=None), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=True, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, union_control_type=<ControlNetUnionControlType.UNKNOWN: 'Unknown'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[], batch_keyframe_idx=None), None, False, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, False, True, 1, 0, 0, False, 0.5, True, False, 'CUDA', False, 0, 'None', '', None, False, False, 0.5, 0, 'tab_single', '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, 'start', '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "C:\SD\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\SD\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\SD\modules\img2img.py", line 235, in img2img
        processed = process_images(p)
      File "C:\SD\modules\processing.py", line 785, in process_images
        res = process_images_inner(p)
      File "C:\SD\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 59, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\SD\modules\processing.py", line 921, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\SD\modules\processing.py", line 1661, in sample
        samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
      File "C:\SD\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\SD\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\SD\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\SD\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\SD\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\SD\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\SD\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\SD\modules\sd_samplers_cfg_denoiser.py", line 269, in forward
        devices.test_for_nans(x_out, "unet")
      File "C:\SD\modules\devices.py", line 255, in test_for_nans
        raise NansException(message)
    modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.
RomanS2N commented 4 months ago

I think there is a conflict with some module ...

light-and-ray commented 3 months ago

Thanks for reporting. Earlier I used lama from controlnet, but now I use from spandrel library, which is used also for upscalers in sd-webui

I can't reproduce the error, maybe we have different versions of the webui or the library

I've commited a fix, can you test it?

light-and-ray commented 3 months ago

Sorry, I've mixed up branches. Now the patch is in master

RomanS2N commented 3 months ago

Again: modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

image

Can you make up branches on old version?

RomanS2N commented 3 months ago

If I enabled "Upcast cross attention layer to float32" or not, it still doesn't work!

RomanS2N commented 3 months ago

If you need any more information, feel free to let me know!

RomanS2N commented 3 months ago

AttributeError: 'MaskedImageModelDescriptor' object has no attribute 'cpu' - diden't displayed ...

RomanS2N commented 3 months ago

so I started using optimazed "bat" file with next keys:

set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:24
set COMMANDLINE_ARGS= --opt-sdp-attention --upcast-sampling --no-hashing --always-batch-cond-uncond --no-half-vae 

It is working? but only in first time showed som error:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\SD\venv\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 240, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "C:\SD\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\gzip.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "C:\SD\venv\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "C:\SD\venv\lib\site-packages\gradio\routes.py", line 604, in join_queue
    session_info = await asyncio.wait_for(
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 445, in wait_for
    return fut.result()
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 133, in receive_json
    self._raise_on_disconnect(message)
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 105, in _raise_on_disconnect
    raise WebSocketDisconnect(message["code"])
starlette.websockets.WebSocketDisconnect: 1001
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\SD\venv\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 240, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "C:\SD\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\gzip.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "C:\SD\venv\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "C:\SD\venv\lib\site-packages\gradio\routes.py", line 604, in join_queue
    session_info = await asyncio.wait_for(
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 445, in wait_for
    return fut.result()
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 133, in receive_json
    self._raise_on_disconnect(message)
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 105, in _raise_on_disconnect
    raise WebSocketDisconnect(message["code"])
starlette.websockets.WebSocketDisconnect: 1001
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\SD\venv\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 240, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "C:\SD\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\base.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\gzip.py", line 26, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\SD\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "C:\SD\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "C:\SD\venv\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "C:\SD\venv\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "C:\SD\venv\lib\site-packages\gradio\routes.py", line 604, in join_queue
    session_info = await asyncio.wait_for(
  File "C:\Users\Roman\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 445, in wait_for
    return fut.result()
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 133, in receive_json
    self._raise_on_disconnect(message)
  File "C:\SD\venv\lib\site-packages\starlette\websockets.py", line 105, in _raise_on_disconnect
    raise WebSocketDisconnect(message["code"])
starlette.websockets.WebSocketDisconnect: 1001
light-and-ray commented 3 months ago

Again:

I don't this this error is connected with the extension. It's in stable diffusion pipeline

starlette.websockets.WebSocketDisconnect: 1001

You need to reload frontend page I think

AttributeError: 'MaskedImageModelDescriptor' object has no attribute 'cpu' - diden't displayed ...

Nice, so I think the bug has gone. Can you test in extras tab?

RomanS2N commented 3 months ago

I checked! It works. Interestingly, I can never remove more than 3-5% of the image on the marked area. LAMA draws a big blur spot, by now in "extras" LAMA was able to replace the drawing with one that was correct in meaning. How FIX this?

light-and-ray commented 3 months ago

In inpainting tab blur means that masked content fallbacked to "fill", it can be caused by an error in my extension

I can't reproduce, for me it works in inpaint tab. You can send the full console log again. Also you can try to remove venv directory and reinstall all dependencies

RomanS2N commented 3 months ago

I reinstalled and by now it works ... Thank!