Error on txt2mask grounded_sam

[cian0]

Shortcode is:

[txt2mask mode="add" method="grounded_sam" show] Face [/txt2mask] person

Error running process: /stable-diffusion-webui/extensions/_unprompted/scripts/unprompted.py
Traceback (most recent call last):
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shortcodes.py", line 137, in render
    return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context, content))
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shared.py", line 78, in handler
    return(self.shortcode_objects[f"{keyword}"].run_block(pargs, kwargs, context, content))
  File "/stable-diffusion-webui/extensions/_unprompted/shortcodes/stable_diffusion/txt2mask.py", line 609, in run_block
    self.image_mask = get_mask().resize((self.init_image.width,self.init_image.height))
  File "/stable-diffusion-webui/extensions/_unprompted/shortcodes/stable_diffusion/txt2mask.py", line 447, in get_mask
    predictor = SamPredictor(build_sam(checkpoint=sam_file).to(device))
  File "/media/ian/extras/condaenvs/sd_webui/lib/python3.10/site-packages/segment_anything/build_sam.py", line 15, in build_sam_vit_h
    return _build_sam(
  File "/media/ian/extras/condaenvs/sd_webui/lib/python3.10/site-packages/segment_anything/build_sam.py", line 105, in _build_sam
    state_dict = torch.load(f)
  File "/stable-diffusion-webui/modules/safe.py", line 107, in load
    return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs)
  File "/stable-diffusion-webui/modules/safe.py", line 152, in load_with_extra
    return unsafe_torch_load(filename, *args, **kwargs)
  File "/home/ian/.local/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/ian/.local/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/stable-diffusion-webui/modules/scripts.py", line 451, in process
    script.process(p, *script_args)
  File "/stable-diffusion-webui/extensions/_unprompted/scripts/unprompted.py", line 469, in process
    Unprompted.shortcode_user_vars["prompt"] = Unprompted.process_string(apply_prompt_template(original_prompt,Unprompted.Config.templates.default))
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shared.py", line 96, in process_string
    string = self.shortcode_parser.parse(self.sanitize_pre(string,self.Config.syntax.sanitize_before),context)
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shortcodes.py", line 245, in parse
    return stack.pop().render(context)
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shortcodes.py", line 59, in render
    return ''.join(child.render(context) for child in self.children)
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shortcodes.py", line 59, in <genexpr>
    return ''.join(child.render(context) for child in self.children)
  File "/stable-diffusion-webui/extensions/_unprompted/lib_unprompted/shortcodes.py", line 141, in render
    raise ShortcodeRenderingError(msg) from ex
lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'txt2mask' shortcode in line 1.

[ThereforeGames]

Hi cian0,

I too have encountered occasional issues with grounded_sam and have elected to remove it from Unprompted, in favor of the newer (and seemingly superior) fastsam. Feel free to give it a try and let me know if the issues persist. Thanks!

[cian0]

Nice! Thank you!

[LGlassySky]

Hi cian0,

I too have encountered occasional issues with grounded_sam and have elected to remove it from Unprompted, in favor of the newer (and seemingly superior) fastsam. Feel free to give it a try and let me know if the issues persist. Thanks!

Using the latter two modes of txt2mask will still report an error, the following is my error content,

Error running process: D:\sd-webui-aki-v4.2\extensions\unprompted\scripts\unprompted.py Traceback (most recent call last): File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 144, in render return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context, content)) File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shared.py", line 86, in handler return(self.shortcode_objects[f"{keyword}"].run_block(pargs, kwargs, context, content)) File "D:\sd-webui-aki-v4.2\extensions\unprompted/shortcodes\stable_diffusion\txt2mask.py", line 603, in run_block self.image_mask = get_mask().resize((self.init_image.width,self.init_image.height)) File "D:\sd-webui-aki-v4.2\extensions\unprompted/shortcodes\stable_diffusion\txt2mask.py", line 501, in get_mask preds = model(img.repeat(prompt_parts,1,1,1).to(device=device), prompts)[0].cpu() File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\clip_surgery\clip_surgery_model.py", line 475, in forward text_features = self.encode_text(text) File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\clip_surgery\clip_surgery_model.py", line 459, in encode_text x = self.token_embedding(text).type(self.dtype) # [batch_size, n_ctx, d_model] File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\modules\sparse.py", line 162, in forward return F.embedding( File "D:\sd-webui-aki-v4.2\python\lib\site-packages\torch\nn\functional.py", line 2210, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not list 提示：Python 运行时抛出了一个异常。请检查疑难解答页面。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\sd-webui-aki-v4.2\modules\scripts.py", line 474, in process
    script.process(p, *script_args)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\scripts\unprompted.py", line 550, in process
    Unprompted.shortcode_user_vars["prompt"] = Unprompted.process_string(apply_prompt_template(Unprompted.original_prompt,Unprompted.Config.templates.default))
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shared.py", line 104, in process_string
    string = self.shortcode_parser.parse(self.sanitize_pre(string,self.Config.syntax.sanitize_before),context)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 252, in parse
    return stack.pop().render(context)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 59, in render
    return ''.join(child.render(context) for child in self.children)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 59, in <genexpr>
    return ''.join(child.render(context) for child in self.children)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 142, in render
    content = ''.join(child.render(context) for child in self.children)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 142, in <genexpr>
    content = ''.join(child.render(context) for child in self.children)
  File "D:\sd-webui-aki-v4.2\extensions\unprompted\lib_unprompted\shortcodes.py", line 148, in render
    raise ShortcodeRenderingError(msg) from ex
lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'txt2mask' shortcode in line 1.

---

[djbielejeski]

I also get the same errors while using either fastsam or clip_surgery. Default works fine

Works

[txt2mask mode="discard" precision=120.0 padding=10.0 smoothing=20.0 neg_precision=80.0 neg_padding=0.0 neg_smoothing=20.0 negative_mask="shirt|clothes|background|arms|collar"]face|forehead|head|hair[/txt2mask]

Errors out [txt2mask method="fastsam" mode="discard" precision=120.0 padding=10.0 smoothing=20.0 neg_precision=80.0 neg_padding=0.0 neg_smoothing=20.0 negative_mask="shirt|clothes|background|arms|collar"]face|forehead|head|hair[/txt2mask]

[txt2mask method="clip_surgery" mode="discard" precision=120.0 padding=10.0 smoothing=20.0 neg_precision=80.0 neg_padding=0.0 neg_smoothing=20.0 negative_mask="shirt|clothes|background|arms|collar"]face|forehead|head|hair[/txt2mask]