Problem of Evaluation

[ZhiyeTang]

Firstly, thanks for your efforts in releasing your code. However, I found something strange in train.py. In line 669, the script tried to recover the latest iteration of GaussianModel, and evaluated its performance. Are the evaluated results consistent with what you reported in your Arxiv paper? It seems these results are based on the parameters without encoding and decoding.

Furthermore, in function generate_neural_gaussians, if the GaussianModel is in a decoded version, the neural gaussians will be directly generated by pc._anchor_feat, which are decoded from a bitstream encoded from pc._anchor_feat. In other words, in the decoded version, it seems the used features are not interpolated by grid_encoder but the original pc._anchor_feat instead.

[YihangChen-ee]

Hi, thanks for your interest!

Firstly, please read our code more carefully and you will find we conduct encoding and decoding in line 300-309, which replaces the anchor parameters with the decoded ones (pls refer to the function conduct_decoding to see this replacement). This encoding and decoding process is conducted at the end of the training process, which is before conducting line 669. Therefore, at the end of training process, the weight we save for GaussianModel is the decoded version; so in line 669, the GaussianModel that we recover is also the same decoded version.

Secondly, I am sorry for any confusion but I am afraid you have some misunderstanding to our method. Once we get hash features, we do not use them to replace pc._anchor_feat, but use them to build the context and model the entropy of pc._anchor_feat. Therefore, once pc._anchor_feat is successfully decoded with the help of grid_encoder in the decoding process (pls refer to the function conduct_decoding to see this decoding process), the grid_encoder is no longer needed, and the pc._anchor_feat itself is exactly the decoded version. I recommend you take a more careful read to our paper.

Best

[ZhiyeTang]

Thank you for your reply. I did ignore something important previously.

As to the second issue I raised, I noticed that in generate_neural_gaussians, the pc._anchor_feat will still be used to predict the corresponding scale_rot and color. That is, during rendering, the feat_context from grid_encoder is just context, assisting mlps to predict the attributes of neural gaussians. Am I understanding correctly?

[YihangChen-ee]

Hi, yes, you are correct.