AMD gpu users can't run inpaint anything

[yacinesh]

@Uminosachi You know already that AMD gpu users when run inpaint anything will get errors, until we activate "Run Segment Anything on CPU" option , but it will take so long to generate So can you find a solution, and thank you

[Uminosachi]

The Segment Anything model is written in PyTorch. Therefore, if it does not run on PyTorch for ROCm on AMD GPUs, it may be necessary to review and revise the model's code. However, making such adjustments could be challenging.

[yacinesh]

@Uminosachi Will you try to do something about it ?

[Uminosachi]

I don't have an AMD GPU card, so I can't verify the error.

[yacinesh]

i will be here for the test if you want to

[Uminosachi]

If you could record the error you encounter when running Segment Anything with PyTorch for ROCm, I'd be willing to look into it.

[yacinesh]

@Uminosachi Here you go

Startup time: 19.6s (prepare environment: 6.5s, import torch: 3.8s, import gradio: 1.1s, setup paths: 1.0s, initialize shared: 1.5s, other imports: 0.3s, load scripts: 3.3s, refresh VAE: 0.4s, create ui: 0.9s, gradio launch: 0.4s, app_started_callback: 0.2s). Creating model from config: C:\a1111\stable-diffusion-webui-directml\configs\v1-inpainting-inference.yaml Applying attention optimization: sub-quadratic... done. Model loaded in 6.8s (load weights from disk: 2.2s, create model: 0.4s, apply weights to model: 2.6s, apply float(): 0.9s, calculate empty prompt: 0.7s). 2024-02-09 17:56:20,376 - Inpaint Anything - INFO - input_image: (253, 450, 3) uint8 2024-02-09 17:56:21,693 - Inpaint Anything - INFO - SamAutomaticMaskGenerator sam_hq_vit_b.pth Traceback (most recent call last): File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\scripts\inpaint_anything.py", line 187, in run_sam sam_masks = inpalib.generate_sam_masks(input_image, sam_model_id, anime_style_chk) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\inpalib\samlib.py", line 139, in generate_sam_masks sam_masks = sam_mask_generator.generate(input_image) File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 151, in generate mask_data = self._generate_masks(image, multimask_output) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 194, in _generate_masks crop_data = self._process_crop(image, crop_box, layer_idx, orig_size, multimask_output) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 245, in _process_crop batch_data = self._process_batch(points, cropped_im_size, crop_box, orig_size, multimask_output) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\automatic_mask_generator.py", line 289, in _process_batch masks, ioupreds, = self.predictor.predict_torch( File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\predictor.py", line 227, in predict_torch sparse_embeddings, dense_embeddings = self.model.prompt_encoder( File "C:\a1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\modeling\prompt_encoder.py", line 155, in forward point_embeddings = self._embed_points(coords, labels, pad=(boxes is None)) File "C:\a1111\stable-diffusion-webui-directml\extensions\sd-webui-inpaint-anything\segment_anything_hq\modeling\prompt_encoder.py", line 89, in _embed_points point_embedding[labels == 0] += self.point_embeddings[0].weight RuntimeError: The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

2024-02-09 17:56:25,944 - Inpaint Anything - ERROR - The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

[Uminosachi]

The size of tensor a (0) must match the size of tensor b (256) at non-singleton dimension 1

I have seen this error before and believe it is due to ROCm's PyTorch not supporting higher dimensional shapes (e.g., length 5). Modifying the model could lead to an inability to load the standard weights, resulting in a significant overhaul that encompasses two models and carries risks.

[yacinesh]

So it's impossible to do something about it ?

[Uminosachi]

Either ROCm's PyTorch needs to support the Segment Anything model, or the developers of the Segment Anything model need to provide support.

[ride5k]

fyi I am able to use with GPU processing on amd 6700xt via zluda on a1111.