reproduce psnr

[baoachun]

Hi, I tried to retrain the model according to the following command, and the test psnr was only 25.33. In addition, I used the CR-NeRF-branden.ckpt model in the cloud disk for testing, and the accuracy was only 24.55. Can you please help me figure out where the problem is with the settings?

Training
python train_mask_grid_sample.py \
    --root_dir ../brandenburg_gate/ \
    --dataset_name phototourism \
    --save_dir ./tmp \
    --img_downscale 2 \
    --N_importance 64 --N_samples 64 \
    --num_epochs 1 --batch_size 1024 \
    --optimizer adam --lr 5e-4 --lr_scheduler cosine \
    --N_emb_xyz 15 --N_vocab 1500 \
    --maskrs_max 5e-2 --maskrs_min 6e-3 \
    --maskrs_k 1e-3 --maskrd 0 --N_a 48 \
    --weightKL 1e-5 --weightRecA 1e-3 \
    --weightMS 1e-6 --chunk 1310720 \
    --encode_a --encode_c  --encode_random \
    --model_mode 1-1 --decoder linearStyle \
    --decoder_num_res_blocks 1 \
    --nerf_out_dim 64 \
    --use_cache --proj_name style_gnerf \
    --use_mask --exp_name train/exp1 \
    --num_gpus 4

Testing
python eval.py \
  --root_dir ../brandenburg_gate/ \
  --save_dir ./tmp \
  --dataset_name phototourism \
  --scene_name train/exp1 \
  --split test_test \
  --img_downscale 2 \
  --N_samples 256 \
  --N_importance 256 \
  --N_emb_xyz 15 \
  --N_vocab 1500 \
  --ckpt_path ./tmp/ckpts/train/exp1/epoch20.ckpt \
  --chunk 20480 --img_wh 320 240 \
  --encode_a --decoder linearStyle \
  --decoder_num_res_blocks 1 \
  --nerf_out_dim 64

[YifYang993]

1.training: According to your training command, you have only trained for 1 epoch. However, In our settings, we train for 20 epoch, please try to set the --num_epochs to 20 instead of 1.

2. I will try that uploaded model, maybe I have uploaded the model trained on downscale=4 (which uses command --img_downscale 4), I will verify that soon.

[YifYang993]

Update on Q2 about testing. just change the --img_downscale from 2 to 4, then, you will get 26.55PSNR

[baoachun]

Thank you for your helping! The above training accuracy is obtained after I trained for 20 epochs. In the command, epoch=1 is typo, I'm sorry about that. In addition, according to the settings you gave, 26.55 still does not match the 26.86 in the paper. How can I reproduce it completely? Because I made some improvements, but I still couldn't perfectly reproduce the accuracy in the paper.

python train_mask_grid_sample.py \
    --root_dir ../brandenburg_gate/ \
    --dataset_name phototourism \
    --save_dir ./tmp \
    --img_downscale 2 \
    --N_importance 64 --N_samples 64 \
    --num_epochs 20 --batch_size 1024 \
    --optimizer adam --lr 5e-4 --lr_scheduler cosine \
    --N_emb_xyz 15 --N_vocab 1500 \
    --maskrs_max 5e-2 --maskrs_min 6e-3 \
    --maskrs_k 1e-3 --maskrd 0 --N_a 48 \
    --weightKL 1e-5 --weightRecA 1e-3 \
    --weightMS 1e-6 --chunk 1310720 \
    --encode_a --encode_c  --encode_random \
    --model_mode 1-1 --decoder linearStyle \
    --decoder_num_res_blocks 1 \
    --nerf_out_dim 64 \
    --use_cache --proj_name style_gnerf \
    --use_mask --exp_name train/exp1 \
    --num_gpus 4

[YifYang993]

OK, about the 26.86 PSNR, I have uploaded the exact checkpoint that get the exact 26.86 PSNR, please download the CR-NeRF-branden.ckpt in the google drive linkhttps://drive.google.com/drive/folders/1yuV8_FtqCPtDN1g8t_4kZaKwoXbTuqZe?usp=sharing

About the training results, I will run your command to see if there is any difference. It will take some time.

[YifYang993]

Hi, I have retrained the model with 8 TITAN Xp GPU, the result is 26.74851417541504 PSNR, which is higher than the result I reported in the paper. Maybe you could try to compare your training log with mine in wandb report (https://wandb.ai/y6216886/style_gnerf/reports/Reproducing-Results-of-CR-NeRF-on-image-downscale-ratio-2--Vmlldzo1OTQyNzY3)

My command is:

nerf_out_dim1=64 exp_name1="train/reproduce-ds2-20231111" model_mode1="1-1" decoder='linearStyle'
ckpt_path1="/mnt/cephfs/dataset/NVS/nerfInWild/experimental_results/ckpts/${exp_name1}/last.ckpt" root_dir1="/mnt/cephfs/dataset/NVS/nerfInWild/brandenburg_gate/" dataset_name1='phototourism' save_dir1=/mnt/cephfs/dataset/NVS/nerfInWild/experimental_results decoder_num_res_blocks=1 img_downscale=2

train

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python train_mask_grid_sample.py --root_dir $root_dir1 --dataset_name phototourism --save_dir /mnt/cephfs/dataset/NVS/nerfInWild/experimental_results --img_downscale $img_downscale --N_importance 64 --N_samples 64 --num_epochs 20 --batch_size 1024 --optimizer adam --lr 5e-4 --lr_scheduler cosine --N_emb_xyz 15 --N_vocab 1500 --maskrs_max 5e-2 --maskrs_min 6e-3 --maskrs_k 1e-3 --maskrd 0 --N_a 48 --weightKL 1e-5 --weightRecA 1e-3 --weightMS 1e-6 --chunk 1310720 --encode_a --encode_c --encode_random --model_mode 1-1 --decoder linearStyle --decoder_num_res_blocks $decoder_num_res_blocks --nerf_out_dim $nerf_out_dim1 --use_cache --proj_name style_gnerf --num_gpus 8 --use_mask --exp_name $exp_name1

test

cd /mnt/cephfs/dataset/NVS/nerfInWild/experimental_results/logs/$exp_name1/codes

render image

CUDA_VISIBLE_DEVICES=3 python eval.py \ --root_dir $root_dir1 \ --save_dir $save_dir1 \ --dataset_name $dataset_name1 --scene_name $exp_name1 \ --split test_test --img_downscale $img_downscale \ --N_samples 256 --N_importance 256 --N_emb_xyz 15 \ --N_vocab 1500 \ --ckpt_path $ckpt_path1 \ --chunk 2048 --img_wh 320 240 --encode_a --decoder $decoder --decoder_num_res_blocks $decoder_num_res_blocks --nerf_out_dim $nerf_out_dim1

calculate metrics

CUDA_VISIBLE_DEVICES=0 python eval_metric.py \ --root_dir $root_dir1 \ --save_dir $save_dir1 \ --dataset_name $dataset_name1 --scene_name $exp_name1 \ --split test_test --img_downscale $img_downscale \ --img_wh 320 240

The log of train and test results:

[baoachun]

I've gotten similar results, thank you very much for your help!