pointrix-project / msplat

A modular differential gaussian rasterization library.
Other
170 stars 10 forks source link

Train high dimensional features #15

Open JoannaCCJH opened 1 month ago

JoannaCCJH commented 1 month ago

Hi! If I want to train and render an additional high-dimensional feature vector for each Gaussian, should I revise the following part in gaussian_render? Besides this part, what else should I do?

render = ms.alpha_blending(
                    uv, conic, opacity, high_dim_feature,
                    gaussian_ids_sorted, tile_range, bg_color[0].item(), width, height, ndc
                )

I tried to render an already-trained Gaussian model with high-dim features, but the rendered features were not as expected.

yGaoJiany commented 1 month ago

If _high_dimfeature has a shape of [P, N, C], it should work. The feature maps will be obtained through alpha blending.

JoannaCCJH commented 1 month ago

If _high_dimfeature has a shape of [P, N, C], it should work. The feature maps will be obtained through alpha blending.

Can you elaborate more on what P, N, C represent respectively? I originally thought the shape should be like [N, C] where N is the number of Gaussian and C is the channel number

yGaoJiany commented 1 month ago

Hmm, I've made an mistake. The correct notation is [P, C], where P represents the point number and C denotes the channel. You're absolutely right.

yGaoJiany commented 1 month ago

Would you be so kind as to provide some data for this?