street数据集中的pose貌似有问题

[SunHongyang10]

您好，我在用您street的数据集来跑3d gaussian，我发现您street数据集中的pose似乎有问题，简单来说，用sfm生成的pose得到的结果比使用您数据集中的pose得到的结果要好很多，我不清楚是不是我错误的使用了您的数据集。

我使用的pose是用您提供的脚本生成的，从pose文件夹中读取的，代码如下： c2w = np.array(frame['transform_matrix']) c2w[:3, 1:3] *= -1 exts.append(c2w) 这个pose，我在front视角和back视角都可以得到一个相对还可以的结果，但是如果是right视角或者是left视角效果就会很差。

下面是我用您的pose得到的结果：

https://github.com/city-super/MatrixCity/assets/85117719/d9b47226-6843-4107-b4eb-69b7433af537

下面是我用sfm生成的pose得到的结果：

https://github.com/city-super/MatrixCity/assets/85117719/5f9f5d2e-d691-418b-9a6a-f95e5eda59db

请问我使用您提供的pose的方式是否错误？目前使用的方式参考了octree gs中pose的读取。

期待您的回答～谢谢～

[jianglh-WHU]

1. You can visualize the point cloud and poses to vertify whether it's correct using https://github.com/jianglh-WHU/preprocess/blob/main/landmark/poses.py
2. You may find guidance in the discussion at https://github.com/city-super/Octree-GS/issues/16, as it might offer insights relevant to your situation.

[SunHongyang10]

1. You can visualize the point cloud and poses to vertify whether it's correct using https://github.com/jianglh-WHU/preprocess/blob/main/landmark/poses.py
2. You may find guidance in the discussion at matrixCity street problem Octree-GS#16, as it might offer insights relevant to your situation.

谢谢～我刚刚尝试了一下把c2w[:3, 3] *= 10，确实会好很多，在right视角达成了比较好的效果，

但是把c2w[:3, 3] *= 10后，在front视角的表现会下降，3d gs拟合的速度也会变慢，远处的建筑重建的就不好（之前不会有这个现象） 实验结果如下：

https://github.com/city-super/MatrixCity/assets/85117719/2459d82c-8ac8-42c8-96a1-272ee915120d

感觉把c2w[:3, 3] *= 10后，近处的拟合好了，但是远处的拟合效果也降低了

[jianglh-WHU]

1. How does it compare with SfM initilization?
2. The speed of 3D-GS is mainly related with the number of Gaussian primitives. If a large number of primitives are pruned at the beginning, the process will speed up, but the visual quality may significantly decrease. Conversely, the quality will be better with more primitives as usual.

[SunHongyang10]

1. How does it compare with SfM initilization?
2. The speed of 3D-GS is mainly related with the number of Gaussian primitives. If a large number of primitives are pruned at the beginning, the process will speed up, but the visual quality may significantly decrease. Conversely, the quality will be better with more primitives as usual.

是的，您说的很对，关于速度方面主要与高斯原语的数量有关。 但目前直观的比较是，如果c2w[:3, 3] *= 10后，front视角，远处的效果没有之前好，但是近处如街道上的涂鸦则会更清晰，因此我好奇，是否pose应该做一些其他的预处理？

[SunHongyang10]

1. How does it compare with SfM initilization?
2. The speed of 3D-GS is mainly related with the number of Gaussian primitives. If a large number of primitives are pruned at the beginning, the process will speed up, but the visual quality may significantly decrease. Conversely, the quality will be better with more primitives as usual.

与sfm的对比实验还在跑，sorry~

[jianglh-WHU]

1. I'm not really sure why the distant viewpoints are blurring out, I don't seem to have this problem when I train with scaffold-gs/octree-gs https://github.com/city-super/Octree-GS/issues/16.
2. The influence of scale is primarily thought to be associated with the learning rate.

[SunHongyang10]

1. How does it compare with SfM initilization?
2. The speed of 3D-GS is mainly related with the number of Gaussian primitives. If a large number of primitives are pruned at the beginning, the process will speed up, but the visual quality may significantly decrease. Conversely, the quality will be better with more primitives as usual.

您好，和sfm init的对比实验已经好了，因为sfm只用front视角没办法find good initial image pair，所以我sfm和c2w[:3, 3]*10都跑了四个视角，下面是实验结果，请您过目： 首先是sfm init的实验结果

https://github.com/city-super/MatrixCity/assets/85117719/d59363f1-2c22-48d1-83c0-b93022eeb8c4

然后是对c2w[:3, 3]*10后的实验结果

https://github.com/city-super/MatrixCity/assets/85117719/7ee26a16-2611-48a2-8cb8-26e3873061c4

[jianglh-WHU]

The result looks so weird, it's the block small scene?

[SunHongyang10]

The result looks so weird, it's the block small scene?

并不是用了block small，而是简单的用了small_city/street/train_dense/small_city_road_down_dense文件夹中的0-60图片作为front视角，然后再从small_city_road_down_dense文件夹中选择对应的left, right, back视角（实际上一张图片的id+771就是同一个位置的不同视角，但仅限于这个序列）

[SunHongyang10]

The result looks so weird, it's the block small scene?

我看到了您的实验结果，效果非常好，请问这是怎么做到的，是算法不同的原因吗？

[jianglh-WHU]

yeah, first you can use scaffold-gs to train, and then you can also utilize the depth attribute to eliminate floaters.

[SunHongyang10]

yeah, first you can use scaffold-gs to train, and then you can also utilize the depth attribute to eliminate floaters.

感谢您的耐心回复，请问您训练scaffold-gs时的设置是什么呢？似乎没有对做c2w[:3, 3]*10的处理？ depth用于监督的话，的确可以有效减少漂浮物，感谢您的建议～

[jianglh-WHU]

You can use the default config first and keep c2w[:3, 3]*10