Concerning the Reproducibility of Results on ScanObjectNN

[aHapBean]

Hi! Thanks for your great job. I find that directly using your repo and your checkpoint of Point-GPT-S cannot reproduce the result in your paper.

Specifically, I download the pretrained.pth of Point-GPT-S and I run your code on downstream benchmark objbg and objonly on ScanobjectNN with minor changes to your code which aim at resolving the environment error.

But I only get 87.95% and 88.12% which is significantly lower than the reported results 91.6% and 91.0% in the paper of PointGPT.

And I think the minor changes to the code will not affect the results. The changes in code include:

From

# tools/builer.py
elif sche_config.type == 'CosLR':
        scheduler = CosineLRScheduler(optimizer,
                                      t_initial=sche_config.kwargs.epochs,
                                      # t_mul=1,
                                      lr_min=1e-6,
                                      cycle_decay=0.1,  # decay_rate
                                      warmup_lr_init=1e-6,
                                      warmup_t=sche_config.kwargs.initial_epochs,
                                      cycle_limit=1,
                                      t_in_epochs=True)

change to

elif sche_config.type == 'CosLR':
        scheduler = CosineLRScheduler(optimizer,
                                      t_initial=sche_config.kwargs.epochs,
                                      # t_mul=1,
                                      lr_min=1e-6,
                                      # cycle_decay=0.1,  # decay_rate
                                      decay_rate=0.1       # NOTE change here
                                      warmup_lr_init=1e-6,
                                      warmup_t=sche_config.kwargs.initial_epochs,
                                      cycle_limit=1,
                                      t_in_epochs=True)

And from

# models/PointGPT.py
        dim_t = self.temperature ** (2 * torch.div(dim_t, 2, rounding_mode='trunc') / self.num_pos_feats)

change to

        dim_t = self.temperature ** (2 * (dim_t / 2).trunc() / self.num_pos_feats)

I have no idea about how to reproduce your result on ScanobjectNN using Point-GPT-S. Could you help me with this?

Thank you.

[aHapBean]

Upon further searching online, I found that the functions decay_rate and cycle_decay serve the same purpose and should not affect the results. Similarly, the second code change involving trunc might not be related. Can you help me understand why the results are so poor?

Thank you!

[CGuangyan-BIT]

This issue stems from the learning rate specified in the config file; immediate modifications will be made. Thank you.