some small mistakes while getting the sparse folder

[Starryskyswt]

Here I find a small mistake while warping a video of the images_downfactor. It should be corrected as out_dir while it was in_dir at first.Hopefully you can check it again,and reply to me if I Ignored other issues.

[cuteyyt]

Thanks for your interest in our project. Which file do these codes belong to? I'll check this.

[cuteyyt]

Thanks for your interest in our project. Which file do these codes belong to? I'll check this.

I have fixed this in datasets/pre_sparse.py (commit f72c358ca78c2ffe8aa8075e9db91f8e7bc247e6). Thank you!

[Starryskyswt]

Yes，that's the place where I found the mistake.Now I encounter the same issue as the other one has put forward.I use the nerf_liff_data(room dataset)to run pre_sparse.py and get two folders including image(Sorted images from the original dataset）and image4(Image after downsampling 1/4), and next step is to use LLFF to reconstruct cam params.Becuase I didn't install colmap and I found camera.bin from the nerf_liff_data room dataset. My question is that after sorted the image of the dataset,can I use camera. bin directly? Or in other words,Does the order of the images in the dataset correspond one-to-one to camera. bin？Now,I may directly use sparse folder of the dataset and skip the first step. I just wonder if I change the order of the pictures in the dataset(like what pre_sparese.py does)，could I use the raw camera. bin from the dataset?Please do me a favour.

[cuteyyt]

Of course, you can use the camera parameters provided in the original dataset directly. But the file names of different views in the bin file and assigned to the image file may not match neatly. For example, in the room scene from nerf_llff_data, the image name in the images.bin file is _DJI_20200226_143850006.JPG corresponds to the images folder while images_4 uses a png suffix like _DJI_20200226_143850006.png.

To make the pipeline work properly, the best way is to ensure a unique name for each view in every usage. However, some reconstruction schemes used in this project, like NeRF, load camera parameters from file poses_bounds.npy, which omits the view name. In this case, the order determines the correspondence between image files and camera parameters. And as we warp all datasets into the same format, we may also use some sort functions lazily to match cameras and image files, which may be dangerous to use the original camera names.

Please carefully check the correspondence to use the camera parameters provided directly in the datasets. If you have any specific questions, I am willing to help.

[Starryskyswt]

Thanks for your asistance,I have figured out the corresdence between camera parameters and pictures.Becasue the nerf_liff dataset already has contained the necessary sparese folder,I have modified the code in order to directly use the existing sparse folder. Now I encounter another problem when running the run_sam_text.py,but I find the ckpt folder may need to be downloaded somewhere(and your code seemingly doesn't include it.). Befor I realize this issue,I have run the code twice.Here are the results. First I use the ladder and find the progress stuck,then I close the laddder and try it again.It seems that the progress gets stuck again.My question is that is it necessary for me to downlaod the ckpts somewhere and whether the running results goes well (what I need to do is just wait?) I would appreciate it if you could help me figure out the issue. Additionally,I find the ckpt folder may need to be downloaded somewhere similarly in the run_sam_point.py,I have no idea where to download it.(I have downloaded all your code, including the code in the Prior folder link）

[cuteyyt]

Yes, you need to download some pre-trained checkpoints.

SAM: generate masks for different views with points or box prompts. Grounding-DINO: generate bounding boxes for images with text prompts. LaMa: inpainting 2D images with masks.

You can download pre-trained checkpoints and install the necessary packages with their guidance.

You can see from the above picture that scripts using sam ckpts already have specified the ckpts. You can use your path as well. Other paths are similar.

[Starryskyswt]

Thank you for your answer,I have downloaded some pre-trained checkpoints above.But I am not sure it's enough or not,cause I find other checkpoints on the link you gave as follow(eg) I wonder if I need to download it,and where to place (because I notice here the default path relates to it) I am confused about it and sincerely expect your reply.

[cuteyyt]

The following files are necessary, and I may miss something. If you have any problems in the future, we can continue this discussion.

ckpts
├── grounding_dino
│   └── groundingdino_swint_ogc.pth
├── lama
│   └── big-lama
│       ├── config.yaml
│       └── models
│           └── best.ckpt
└── sam
    └── sam_vit_h_4b8939.pth

I think you have downloaded all files. The default path means if you do not pass command line parameters to Python scripts, then that default parameter will be used. You can replace the parameter value with your own path.

[cuteyyt]

The file you underlined in the picture is not a model ckpt file but a config file for Grounding-DINO. I think this file is already contained in the project (GroundingDINO_SwinT_OGC.py).