May be bug: rasterizer reduce feature dimension.

[Riften]

I'm trying training sam feature embedding for SfM data sets for Tanks&Temples and Deep Blending from gaussian-splatting. I use provided vit_h pretrained model with speedup cnn. vit_h produces 256-dim feature map, which is used as ground truth during training. However, I got unmatched tensor dimension error for convolution operation during training.

RuntimeError: Given groups=1, weight of size [256, 64, 1, 1], expected input[1, 8, 42, 64] to have 64 channels, but got 8 channels instead

It seems that it is because the rasterizer operation during rendering reduces feature dimension.

• Feature dimension from vit_h: 256-dim
• Supposed input dimension to cnn: 64-dim
• Feature map in gaussians: 64-dim
• Feature map after rasterizer: 8-dim

Is this a bug or wrong configuration?

[ShijieZhou-UCLA]

Hi @Riften! I would suggest a double check for your configuration according to https://github.com/ShijieZhou-UCLA/feature-3dgs?tab=readme-ov-file#train

e.g. If you want to render 64-dim feature and use to speed-up module CNN, you should set:

• NUM_SEMANTIC_CHANNELS = 64 in config.h
• NUMBER = 4 in semantic_feature_size/NUMBER in scene/gaussian_model.py in line 142.
• NUMBER = 4 in feature_out_dim/NUMBER in train.py in line 51.
• NUMBER = 4 in feature_out_dim/NUMBER in render.py in line 117 and 261.