RuntimeError: CUDA error: an illegal memory access was encountered while setting image for predictor

[MaxTeselkin]

Hi! I am trying to use Segment Anything, but I constantly get the following error: RuntimeError: CUDA error: an illegal memory access was encountered.

Here is a small piece of my code:

# build model
sam = sam_model_registry[model_name](checkpoint=weights_path)
# load model on device
torch.cuda.set_device(0)
torch_device = torch.device(device)
sam.to(device=torch_device)
predictor = SamPredictor(sam)
# calculate image embeddings
predictor.set_image(input_image)

Here is where I get this error:

File \"/app/repo/src/main.py\", line 150, in set_image_data
    predictor.set_image(input_image)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/predictor.py\", line 60, in set_image
    self.set_torch_image(input_image_torch, image.shape[:2])
File \"/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py\", line 27, in decorate_context
    return func(*args, **kwargs)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/predictor.py\", line 89, in set_torch_image
    self.features = self.model.image_encoder(input_image)
File \"/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py\", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/modeling/image_encoder.py\
    x = blk(x)
File \"/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py\", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/modeling/image_encoder.py\", line 174, in forward
    x = self.attn(x)
File \"/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py\", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/modeling/image_encoder.py\", line 234, in forward
    attn = add_decomposed_rel_pos(attn, q, self.rel_pos_h, self.rel_pos_w, (H, W), (H, W))
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/modeling/image_encoder.py\", line 349, in add_decomposed_rel_pos
    Rh = get_rel_pos(q_h, k_h, rel_pos_h)
File \"/usr/local/lib/python3.8/dist-packages/segment_anything/modeling/image_encoder.py\", line 322, in get_rel_pos
    return rel_pos_resized[relative_coords.long()]
RuntimeError: CUDA error: an illegal memory access was encountered

I made some research on this issue and know that this issue can be caused by having not enough free GPU memory. I run nvidia-smi command and checked that I have enough GPU memory, but I am still facing this issue. My device is 12 GB Tesla K80.

[mishra077]

Which pytorch version are you using?

[MaxTeselkin]

@mishra077 I am using pytorch 1.12.1

[kalpa61]

@MaxTeselkin Hello, have you solved this problem

[Yo-AvA]

@MaxTeselkin hi! Have you found any solutions?

[NormXU]

I came across the same issue with SamAutomaticMaskGenerator and resolved it by decreasing the points_per_batch.