mask has Y offset, why cause that ?

[LightSun]

Hello , you are write a good project of SAM. I'd like it. But i met a problem: mask has Y offset. can you help me ?

platform

• ubuntu 18

  model

• vit_h

  c++ envs:

• onnx runtime: 1.15.1
• opencv: 4.7.0

  py envs:

• segment-anything 1.0
• onnx 1.14.1
• onnxruntime-gpu 1.16.0

[dinglufe]

Perhaps it's a display issue with OpenCV (clicking the close button might solve it).

Have you tried testing it using the API instead of the executable demo?

[LightSun]

@dinglufe I compile your lib self, Why did you say may be display issue with OpenCV? SAM demo I tried online, but mask don't have offset. I tried to read you c++ code and didn't find 'Which cause the offset '.

[LightSun]

@dinglufe I found the SAM script of export onnx have this.

 dummy_inputs = {
        "image_embeddings": torch.randn(1, embed_dim, *embed_size, dtype=torch.float),
        "point_coords": torch.randint(low=0, high=1024, size=(1, 5, 2), dtype=torch.float),
        "point_labels": torch.randint(low=0, high=4, size=(1, 5), dtype=torch.float),
        "mask_input": torch.randn(1, 1, *mask_input_size, dtype=torch.float),
        "has_mask_input": torch.tensor([1], dtype=torch.float),
        "orig_im_size": torch.tensor([1500, 2250], dtype=torch.float),
    }

But your 'export_pre_module.py' assign the image size 'image_size = (1024, 720)'. Is this the cause ?

[LightSun]

@dinglufe I fixed it by change image_size