InternLandMark / LandMark

Other
441 stars 39 forks source link

The testing set is included in the training set! #17

Closed jay757425789 closed 11 months ago

jay757425789 commented 1 year ago

Hello, I find that the testing set is included in the training set. Meanwhile, the experimental results decreased seriously. In ray_utils.py: if split == "train": with open(os.path.join(root_dir, f'{"transforms_test.json"}'), "r", encoding="utf-8") as f: test_meta = json.load(f) meta["frames"] += test_meta["frames"] this code invalidate the code for data split in city_dataset.py: test_idx = list(range(0, N, self.N_vis)) # test_idx = [0, self.N_vis1, ..., self.N_vis(N//self.N_vis)] train_idx = [i for i in range(N) if i not in test_idx] I have print the image name of testing and training set and find the testing images are almost included in the training set.

eveneveno commented 1 year ago

Thanks for the catch! The initial motivation to first unifying the train and test set is to accomondate the commonly used data format, while for drone captured data, the dense grid pattern is critical for training the entire scene with enough view coverage. And then divide the train/test according to N_vis.

Did you try commenting out if split == "train": (https://github.com/InternLandMark/LandMark/blob/main/app/tools/dataloader/ray_utils.py#L94). Could you also show some results of your failur cases?

Saaaaln commented 11 months ago

The latest code has fixed the issue. v0.2