Cannot generate any meaningful output

[jackshanson]

Hi there, First of all, thank you for making this repo. I've found it very easy to at least run and tinker with, but unfortunately I am having a great deal of trouble generating useful outputs from this code and I've been scratching my head trying to understand where I have gone wrong. I have tried for multiple objects (pineapples, cars, gum containers, forward-facing and 360 deg. views, frames extracted from video and individual stills, and anywhere from 30-150 frames!) with almost no luck. I have followed the tips in #8 and #50 (my shaky hand isn't taking exactly-level photos...) to no avail.

The parameters for a recent experiment are listed below, where I took 360 degree still image views of a bouquet of flowers:

Branch: dev Data: https://drive.google.com/drive/folders/1l6fh7XAvNw1XFNV5Qjv0EfgTw6NSfh-o?usp=sharing Code:

#!/bin/bash

DATASET_PATH=$1
EXP_NAME=$2
SCENE_NAME=$3

colmap feature_extractor --database_path $DATASET_PATH/database.db --image_path $DATASET_PATH/images

colmap sequential_matcher --database_path $DATASET_PATH/database.db

mkdir $DATASET_PATH/sparse

colmap mapper --database_path $DATASET_PATH/database.db --image_path $DATASET_PATH/images --output_path $DATASET_PATH/sparse

python train.py \
   --dataset_name llff \
   --root_dir $DATASET_PATH \
   --N_importance 64 --img_wh 320 240 \
   --num_epochs 60 --batch_size 1024 \
   --optimizer adam --lr 5e-4 \
   --lr_scheduler steplr --decay_step 20 40 --decay_gamma 0.5 \
   --exp_name $EXP_NAME \
   --spheric

CKPT_PATH=`find ckpts/$EXP_NAME/ -type f | sort -n | tail -n 1`

python eval.py \
   --root_dir $DATASET_PATH \
   --dataset_name llff --scene_name $SCENE_NAME \
   --img_wh 320 240 --N_importance 64 --ckpt_path $CKPT_PATH \
   --spheric_poses

note that I tried it also with 30 epochs as per the default instructions, this was just another test run.

Logs:

Outputs (every 2nd frame):

I hope that I have missed something simple, but I don't know where to take it from here. To be honest, I think that my photos are far from optimal and might be the cause of my issues, but I will await your input.

Thanks!

[jackshanson]

Of course as soon as I post this issue, I change something and I get a reasonable output...

I changed sequential_matcher to exhaustive matcher (a remnant from when I tried to process video frames - this never seemed to work) and got the resulting gif:

Strangely, all of my training outputs seem to be the same. Is there any reason why the above would have changed the eval function's output despite me getting a similar PSNR for both training and val data in both runs?

[kwea123]

Hmm, if the only thing that is different for both runs is the matcher, I think that it's a problem from COLMAP's pose estimation. Also you can see that the training PSNR (>26) is far better than val PSNR (<20). Can you visualize the poses using COLMAP's GUI? I'm afraid the sequential matcher one would be wrong.

[jackshanson]

this is the sequential search:

and this is the exhaustive search:

I think, at least... I haven't really used the GUI for colmap so I think I did it right (feature extraction, feature matching (two different ways), and then reconstruction.

[huying163]

祝贺！这个看起来效果很好，我也在复现这个，可以请教下吗

[kwea123]

@huying163 feel free to open another issue.