Jumpat / SegAnyGAussians

The official implementation of SAGA (Segment Any 3D GAussians)
Apache License 2.0
548 stars 40 forks source link

the mip360/garden dataset not work #102

Open blueFeather111 opened 2 weeks ago

blueFeather111 commented 2 weeks ago

我用的v2 branch, 因为cuda内存限制,garden images用的是downsample=8 在分割时click 了花,花盆,桌子,但是不管哪种情况分割出的都是地板。

Screenshot from 2024-08-28 18-09-24

但是从feature上看,它们的特征是不同的,请问这是哪里有问题呢?

Screenshot from 2024-08-28 18-19-28

相关代码:

fig = plt.figure()
plt.title("please click the target")
plt.axis("off")
ax = fig.add_subplot(111)
ax.imshow(view.original_image.permute(1,2,0).detach().cpu().numpy())
cid = fig.canvas.mpl_connect("button_press_event", onclick)
plt.show()
query_index = np.array(coords)[0]

query_index = (
    int(query_index[0] / view.image_height * view.feature_height),
    int(query_index[1] / view.image_width * view.feature_width),
)

normed_features = torch.nn.functional.normalize(scale_conditioned_feature, dim=-1, p=2)
query_feature = normed_features[query_index[0], query_index[1]]

similarity = torch.einsum('C,HWC->HW', query_feature, normed_features) 
plt.subplot(1, 2, 1)
plt.imshow(similarity.detach().cpu().numpy())
plt.subplot(1, 2, 2)
plt.imshow(similarity.detach().cpu().numpy() > 0.75)
plt.show()
Jumpat commented 2 weeks ago

你好,这可能是由于view.image_height,view.feature_height没有和降采样后的图像对其的原因,导致原来输入进去的点的坐标都被映射到了地板上。可以检查下这两个变量。