Alignment with Truck GT in TnT supplied dataset

[yanivw12]

Hi,

I downloaded the pre-processed TnT dataset that you supplied, and ran into an issue: If I take the sparse COLMAP points and apply the transformation from the Tanks and Temples official site (*_trans.txt), the transformed points should align with the GT point cloud. This is the case for Barn for example:

This also works for the other scans. However, for Truck, and only for Truck, they do not align:

I'm taking TNT_GOF/TrainingSet/Truck/sparse/0/points3D.ply as the sparse COLMAP points, and using the Truck_trans.txt and Truck.ply files from the official TnT site.

I'd appreciate your help in identifying the reason for this - am I missing something?

Thanks in advance.

[hbb1]

Hi, does the truck data work normally by running the evaluation?

[yanivw12]

I'm running the official TnT evaluation code on a mesh that aligns perfectly with the above points3D file, and I input the Truck_COLMAP_SfM.log file that you supplied. The result is a low score because the mesh and GT aren't aligned. This is the precision.ply I get from the TnT evaluation:

As you can see, because the transformed mesh and GT do not align, the crop is incorrect and therefore the metrics are very low.

[yanivw12]

Sorry, clicked on close issue by mistake

[hbb1]

I think you input the incorrect transformation file.

[yanivw12]

This is the transformation I'm using:

It matches with the transformation from the TnT site:

1.138071816345332055e+00 -7.300598829132702583e-02 6.497527927834608752e-01 -2.323759808085589018e+00 -6.536992969315738033e-01 -9.985495936658358995e-02 1.133764632318774668e+00 1.657361383604702088e+00 -1.363067303782293342e-02 -1.306680962583968819e+00 -1.229434503706130222e-01 -7.163783983342566941e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00

When I run the same code for other scans, I get a perfect alignment.

[hbb1]

Did the evaluation for other scans work normally?

[yanivw12]

Yes. For example, precision.ply on Ignatius:

[hbb1]

How about the quantitative results?

[yanivw12]

I get this, which is obviously lower than it should be:

And again, the reconstructed mesh from 2dgs (yellow) after transformation+crop and the gt mesh don't align:

I noticed that the COLMAP poses for Truck that you supplied are quite different from the SfM log on the TnT website, or even from the COLMAP poses I got when running it directly on the source images:

Left is your supplied SfM, right is from the TnT website.

I'd appreciate if you could share how you pre-processed the images to get the COLMAP poses - that'll help me debug the issue.

[hbb1]

Hi. I want to know, is the quantitative result look normally by running our provided script? Does it also faces the same issue? Due to the storage limit, we use the data preprocessed and shared by GOF, which you can refer to. But, early, we have processed the data by ourselves, by running a script here, which should also work normally.

[yanivw12]

The results I showed in my previous message were from running your provided script. The issue is with the original COLMAP poses, something seems to be wrong with them, and I'm still trying to figure out what. Also, when you evaluated it for your paper, you got better results - did you use the same COLMAP poses? do you have the precision and recall ply files to check if they're aligned?

[hbb1]

Hi, I use the data and evaluation script to reproduce the results. I have a double check but found there is no issue. Perhaps there is an issue with your downloaded files or running commands.

[hbb1]

Do other scans, except the Truck, work normally? I need more details to figure out the issue.

[hbb1]

The only reason I can think of is that you supplied a wrong COLMAP_SfM.log file. The file should have been included in TNT_GOF/TrainingSet/{scene}

[yanivw12]

I double checked, and I'm supplying the correct SfM file. I downloaded the TNT_GOF dataset again and got the same file: Can you confirm that this is the same SfM file you're using?

[hbb1]

Do other scans, except the Truck, work normally? I need more details to figure out the issue.

I cannot access to my server right now. BTW, can you answer this question?

[yanivw12]

Yes, for example, Ignatius scan works perfectly fine and I get the expected results: (Yellow and grey meshes are aligned)

[hbb1]

Okay, then the Truck data must have an issue with its COLMAP_SfM.log file. I am not sure if the data is modified. If so, there should be an issue with GOF https://github.com/autonomousvision/gaussian-opacity-fields/blob/main/scripts/run_tnt.py#L39

Because our server has been broken down, I cannot have an immediate check. If you need quick feedback, I suggest opening issue under GOF's repo.

[yanivw12]

Ok, thanks, I'll open an issue there.