The test results of the pre-trained model are inconsistent with the benchmark

[glorioushonor]

Hello, I have successfully tested the pre-trained model, but I found that the results do not match the data in the benchmark, especially for CAPE-NFP. Is this normal? Looking forward to your reply.

[River-Zhang]

It seems that you're using the whole cape dataset (containing 150 scans) rather than cape-NFP (containing 100 scans). Besides we did not refer to cape-easy or cape-hard. We just sum the results and calculate the average. Here is the testing result for the whole cape dataset. You can test again on CAPE-NFP (I guess it's in pose.txt?)

[glorioushonor]

It seems that you're using the whole cape dataset (containing 150 scans) rather than cape-NFP (containing 100 scans). Besides we did not refer to cape-easy or cape-hard. We just sum the results and calculate the average. Here is the testing result for the whole cape dataset. You can test again on CAPE-NFP (I guess it's in pose.txt?)

In my opinion, the contents of pose.txt are consistent with the last 100 data entries in all.txt. The first fifty entries in all.txt are cape-easy, which is essentially CAPE-FP; the last one hundred entries in all.txt are cape-hard, which is essentially CAPE-NFP.

accu_outputs = accumulate(
            outputs,
            rot_num=3,
            split={
                "cape-easy": (0, 50),
                "cape-hard": (50, 150)
            },
        )

[River-Zhang]

Let me check out this

[River-Zhang]

I tested on CAPE-NFP solely and the result is and the whole cape dataset

It seems that I might have released an old version ckpt. I'll change to the correct one as soon as possible.

[River-Zhang]

I have updated the ckpt (https://1drv.ms/u/s!AtYZWFVkmRdwnEDF5XGtfLVzgLsH?e=bpgsji). You can test this again. Thanks very much!

[glorioushonor]

I have updated the ckpt (https://1drv.ms/u/s!AtYZWFVkmRdwnEDF5XGtfLVzgLsH?e=bpgsji). You can test this again. Thanks very much!

@River-Zhang Thanks. The test results of the pre-trained model are exactly the same as yours. However, the results of my own training are very poor, and I don't know why. Was your pre-trained model only trained for 10 epochs?

[River-Zhang]

Yes, it is trained only 10 epochs.