How to choose the variacne of 2D Mip Filter ?

[garrisonz]

Thanks for your great work! I understand more about 3dgs from your work!

In 6.1 Implementation, We choose the variance of our 2D Mip filter as 0.1, approximating a single pixel

My questions: How to deduce from a single pixel to 0.1 value？ By the way, what is the unit of 0.1?

In my understande: In computeCov2D function, cov[0][0] += kernel_size; means the scale (or unit) of kernel_size and cov should be the same, because they can add together. cov is the covariance in ray space. So the unit of 0.1 should be the unit in ray space. But why 0.1 in $(x_0, x_1)$ plane of ray space occupies a aingle pixel?

From Fig. 8 ray space. of EWA Splatting

Thanks!

[chensiyuan030105]

[niujinshuchong]

Hi, the variance of the 2D Gaussian filter is choose to approximate a box filter with a pixel size. We choose the value of 0.1. Here is a figure showing a box filter with 1 pixel wide and three Gaussian filters with different variances: 0.1, 0.3, 1.

[f-dy]

In case anyone's interested, here are benchmarks with kernel size = 0.1 vs 0.3

0.1 is a clear winner on synthetic, but much less on real data

benchmark_nerf_synthetic_ours_mtmt: Multi-scale Training and Multi-scale Testing on the Mip-NeRF 360 dataset

PSNR:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	37.565	27.765	34.745	39.169	35.230	31.988	37.678	32.719	34.607
ks03	35.324	26.999	32.910	37.729	32.716	30.527	35.553	31.368	32.891

SSIM:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	0.991	0.963	0.990	0.991	0.988	0.979	0.994	0.933	0.979
ks03	0.986	0.958	0.986	0.988	0.980	0.975	0.991	0.928	0.974

LPIPS:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	0.009	0.031	0.009	0.010	0.011	0.018	0.005	0.059	0.019
ks03	0.017	0.040	0.015	0.015	0.023	0.023	0.009	0.069	0.026

Count:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	225042	346217	201422	162532	277335	269540	376341	428152	285822
ks03	142865	169266	94122	108496	168791	157234	145845	234104	152590

benchmark_nerf_synthetic_ours_stmt: Single-scale Training and Multi-scale Testing on the Mip-NeRF 360 dataset

PSNR:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	35.615	26.463	32.998	36.141	32.853	30.112	31.713	29.704	31.950
ks03	32.281	25.004	31.185	32.932	29.874	27.744	27.889	27.251	29.270

SSIM:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	0.988	0.958	0.988	0.987	0.983	0.975	0.986	0.922	0.973
ks03	0.976	0.943	0.982	0.977	0.966	0.962	0.965	0.905	0.959

LPIPS:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	0.013	0.035	0.012	0.013	0.016	0.019	0.015	0.068	0.024
ks03	0.025	0.050	0.019	0.019	0.033	0.025	0.023	0.081	0.034

Count:		chair	drums	ficus	hotdog	lego	materials	mic	ship	Average
orig	267065	342581	187353	193614	295019	240818	409766	442234	297306
ks03	215105	271186	166316	184540	240591	191612	340624	398384	251044

benchmark_360v2_ours: Multi-scale Training and Multi-scale Testing on the the Blender dataset

PSNR:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	25.904	22.062	27.973	27.141	22.689	31.890	29.288	31.770	32.572	27.921
ks03	25.867	22.038	27.883	27.190	22.628	31.879	29.357	31.761	32.370	27.886

SSIM:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	0.804	0.656	0.884	0.802	0.655	0.933	0.920	0.936	0.952	0.838
ks03	0.803	0.653	0.882	0.802	0.658	0.933	0.921	0.936	0.951	0.838

LPIPS:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	0.161	0.267	0.090	0.181	0.269	0.175	0.166	0.107	0.157	0.175
ks03	0.167	0.276	0.096	0.183	0.276	0.175	0.166	0.107	0.159	0.178

Count:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	7797092	4317844	5594857	5742251	5045895	2064765	1479042	2143430	1603448	3976513
ks03	7114036	3902664	4777574	5452444	4522585	2022989	1457203	2089632	1598331	3659717

benchmark_360v2_ours_stmt: Single-scale Training and Multi-scale Testing on the the Blender dataset

PSNR:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	27.564	23.846	29.842	28.045	24.128	33.534	30.549	34.144	33.698	29.483
ks03	27.264	23.564	29.664	27.936	24.164	33.294	30.242	33.600	33.107	29.204

SSIM:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	0.871	0.753	0.931	0.847	0.743	0.966	0.946	0.975	0.973	0.889
ks03	0.864	0.744	0.925	0.844	0.744	0.965	0.941	0.972	0.969	0.885

LPIPS:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	0.103	0.190	0.050	0.129	0.196	0.047	0.056	0.027	0.032	0.092
ks03	0.116	0.211	0.057	0.139	0.209	0.047	0.062	0.030	0.037	0.101

Count:		bicycle	flowers	garden	stump	treehill	room	counter	kitchen	bonsai	Average
orig	5405063	3019078	2621019	4474809	4254862	1213345	921229	1286991	1286176	2720285
ks03	4632338	2595209	2133358	4065068	3714773	1065752	821504	1251445	1170510	2383328