search

chenhsuanlin / bundle-adjusting-NeRF

BARF: Bundle-Adjusting Neural Radiance Fields 🤮 (ICCV 2021 oral)
MIT License
788 stars 114 forks source link

Cannot reproduce results on llff:fern #25

Closed Yannnnnnnnnnnn closed 2 years ago

Yannnnnnnnnnnn commented 2 years ago

Thanks for your sharing of codes.

I followed all your tips in the ReadME and tried to reproduce the results of your paper. However, the rotation error of my test is much higher than yours in the paper.

llff:fern rotation translation( x100 )
paper 0.191 0.192
reproduce 0.689 0.193

Can you give some advice? Besides, the depth map and the rendered RGB seem good. image

Thanks!

Yannnnnnnnnnnn commented 2 years ago

Add another experiment:

llff:flower PSNR SSIM LPIPS rotation translation( x100 )
reproduce 24.20 0.71 0.21 1.462 0.239
paper 23.37 0.698 0.211 0.251 0.224
chenhsuanlin commented 2 years ago

Hi @Yannnnnnnnnnnn, I don't have a definite answer for this, but there are various factors (e.g. updated library implementations under the hood) that could affect the final performance (even with fixed seeds), especially with this kind of optimization-based frameworks. Could I ask what PyTorch and CUDA versions you have?

Unfortunately I do not have access to my previous machines anymore, but here are the results I get in my current environment (PyTorch 1.11 and CUDA 11.6) from the exact training/evaluation commands in the README: PSNR SSIM LPIPS rot. error trans. error
fern 23.90 0.71 0.31 0.288 0.199
flower 23.71 0.71 0.20 0.042 0.231

These numbers are pretty consistent with what we reported, although the rotation error does have a higher variance.

Yannnnnnnnnnnn commented 2 years ago

Thanks for your reply.

My current environment is there:

CUDA 11.1
python 3.8.0
pytorch 1.10.1
torch-vision 0.11.2
numpy 1.21.2
opencv-python 4.5.5.62

Can you share your requirment.txt so that I can align my environment with yours?
Currently, I use https://github.com/chenhsuanlin/bundle-adjusting-NeRF/blob/main/requirements.yaml which only requires 'pytorch>=1.9.0'.

Besides, I have done twice experiments on the llff:fern and got almost the same results. Maybe the environment is quite essential to reproduce your results.

Thanks again for your help!

chenhsuanlin commented 2 years ago

Here is the installation file of my current conda environment:

name: barf-env
channels:
  - conda-forge
  - pytorch
dependencies:
  - numpy=1.22.3
  - scipy=1.8.0
  - tqdm=4.63.0
  - termcolor=1.1.0
  - easydict=1.9
  - imageio=2.16.1
  - ipdb=0.13.9
  - pytorch=1.11.0
  - torchvision=0.12.0
  - tensorboard=2.8.0
  - visdom=0.1.8.9
  - matplotlib=3.5.1
  - scikit-video=1.1.11
  - trimesh=3.10.2
  - pyyaml=5.4.1
  - pip=22.0.4
  - gdown=4.4.0
  - pip:
    - lpips==0.1.4
Yannnnnnnnnnnn commented 2 years ago

Thanks, @^_^!

I will try your environment.

Yannnnnnnnnnnn commented 2 years ago

I tried your environment but still got a higher rotation error. I had no idea what happened. Can you give me suggestions so that I can lower the rotation error?

Thanks.

results

resuming from epoch None (iteration 200000)
--------------------------
rot:      1.155
trans:    0.00243*100 = 0.243
--------------------------
--------------------------                                                     
PSNR:     23.66                                                                
SSIM:      0.70
LPIPS:     0.21
--------------------------

train.sh and eval.sh

CUDA_VISIBLE_DEVICES=0 python train.py \
                            --group=llff \
                            --model=barf \
                            --yaml=barf_llff \
                            --name=flower \
                            --data.scene=flower \
                            --barf_c2f=[0.1,0.5]

CUDA_VISIBLE_DEVICES=0 python evaluate.py \
                                --group=llff \
                                --model=barf \
                                --yaml=barf_llff \
                                --name=flower \
                                --data.scene=flower \
                                --resume \

code log

commit 803291bd0ee91c7c13fb5cc42195383c5ade7d15 (HEAD -> main, origin/main, origin/HEAD)
Author: Chen-Hsuan Lin <chenhsuanl@nvidia.com>
Date:   Wed Apr 6 01:18:35 2022 -0700

    validate from Procrustes aligned poses instead

code status

git status
位于分支 main
您的分支与上游分支 'origin/main' 一致。

尚未暂存以备提交的变更：
  （使用 "git add <文件>..." 更新要提交的内容）
  （使用 "git checkout -- <文件>..." 丢弃工作区的改动）

        修改：     requirements.yaml (modified)

未跟踪的文件(untracked file):
  （使用 "git add <文件>..." 以包含要提交的内容）

        data/llff/
        data/nerf_llff_data.zip
        eval.sh
        output/
        train.sh

environment

# packages in environment at /home/yqs/anaconda3/envs/barf-env:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
_openmp_mutex             4.5                       1_gnu    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
absl-py                   1.0.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
aiohttp                   3.8.1            py39hb9d737c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
aiosignal                 1.2.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
alsa-lib                  1.2.3                h516909a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
aom                       3.3.0                h27087fc_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
asttokens                 2.0.5              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
async-timeout             4.0.2              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
backports                 1.0                        py_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
beautifulsoup4            4.10.0             pyha770c72_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
blas                      1.0                         mkl    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
blinker                   1.4                        py_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
brotli                    1.0.9                h166bdaf_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
brotli-bin                1.0.9                h166bdaf_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
brotlipy                  0.7.0           py39hb9d737c_1004    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
bzip2                     1.0.8                h7f98852_4    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
c-ares                    1.18.1               h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ca-certificates           2021.10.8            ha878542_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
cachetools                5.0.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
certifi                   2021.10.8        py39hf3d152e_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
cffi                      1.15.0           py39h4bc2ebd_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
click                     8.1.2            py39hf3d152e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
cryptography              36.0.2           py39hd97740a_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
cudatoolkit               11.3.1               h2bc3f7f_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cycler                    0.11.0             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
dbus                      1.13.6               h5008d03_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
easydict                  1.9                        py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
executing                 0.8.3              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
expat                     2.4.8                h27087fc_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ffmpeg                    5.0.1                h594f047_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
filelock                  3.6.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
fontconfig                2.14.0               h8e229c2_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
fonts-conda-ecosystem     1                             0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
fonts-conda-forge         1                             0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
fonttools                 4.31.2           py39hb9d737c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
freetype                  2.10.4               h0708190_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
frozenlist                1.3.0            py39hb9d737c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gdown                     4.4.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gettext                   0.19.8.1          h73d1719_1008    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
giflib                    5.2.1                h36c2ea0_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gmp                       6.2.1                h58526e2_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gnutls                    3.6.13               h85f3911_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
google-auth               2.6.2              pyh6c4a22f_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
grpcio                    1.45.0           py39h0f497a6_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gst-plugins-base          1.20.1               hcf0ee16_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
gstreamer                 1.20.1               hd4edc92_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
icu                       69.1                 h9c3ff4c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
idna                      3.3                pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
imageio                   2.16.1             pyhcf75d05_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
importlib-metadata        4.11.3           py39hf3d152e_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
intel-openmp              2022.0.1          h06a4308_3633    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ipdb                      0.13.9             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ipython                   8.2.0            py39hf3d152e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
jbig                      2.1               h7f98852_2003    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
jedi                      0.18.1           py39hf3d152e_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
jpeg                      9e                   h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
keyutils                  1.6.1                h166bdaf_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
kiwisolver                1.4.2            py39hf939315_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
krb5                      1.19.3               h3790be6_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lame                      3.100             h7f98852_1001    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lcms2                     2.12                 hddcbb42_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lerc                      3.0                  h9c3ff4c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libblas                   3.9.0            14_linux64_mkl    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libbrotlicommon           1.0.9                h166bdaf_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libbrotlidec              1.0.9                h166bdaf_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libbrotlienc              1.0.9                h166bdaf_7    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libcblas                  3.9.0            14_linux64_mkl    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libclang                  13.0.1          default_hc23dcda_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libdeflate                1.10                 h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libdrm                    2.4.109              h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libedit                   3.1.20191231         he28a2e2_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libevent                  2.1.10               h9b69904_4    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libffi                    3.4.2                h7f98852_5    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libgcc-ng                 11.2.0              h1d223b6_14    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libgfortran-ng            11.2.0              h69a702a_14    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libgfortran5              11.2.0              h5c6108e_14    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libglib                   2.70.2               h174f98d_4    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libgomp                   11.2.0              h1d223b6_14    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libiconv                  1.16                 h516909a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
liblapack                 3.9.0            14_linux64_mkl    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libllvm13                 13.0.1               hf817b99_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libnsl                    2.0.0                h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libogg                    1.3.4                h7f98852_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libopenblas               0.3.20          pthreads_h78a6416_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libopus                   1.3.1                h7f98852_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libpciaccess              0.16                 h516909a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libpng                    1.6.37               h21135ba_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libpq                     14.2                 hd57d9b9_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libprotobuf               3.20.0               h6239696_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libsodium                 1.0.18               h36c2ea0_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libstdcxx-ng              11.2.0              he4da1e4_14    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libtiff                   4.3.0                h542a066_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libuuid                   2.32.1            h7f98852_1000    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libuv                     1.43.0               h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libva                     2.14.0               h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libvpx                    1.11.0               h9c3ff4c_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libwebp                   1.2.2                h3452ae3_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libwebp-base              1.2.2                h7f98852_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libxcb                    1.13              h7f98852_1004    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libxml2                   2.9.12               h885dcf4_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libzlib                   1.2.11            h166bdaf_1014    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
lpips                     0.1.4                    pypi_0    pypi
lz4-c                     1.9.3                h9c3ff4c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
markdown                  3.3.6              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
matplotlib                3.5.1            py39hf3d152e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
matplotlib-base           3.5.1            py39h2fa2bec_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
mkl                       2022.0.1           h06a4308_117    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
multidict                 6.0.2            py39hb9d737c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
mysql-common              8.0.28               haf5c9bc_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
mysql-libs                8.0.28               h28c427c_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ncurses                   6.3                  h9c3ff4c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
nettle                    3.6                  he412f7d_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
nspr                      4.32                 h9c3ff4c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
nss                       3.77                 h2350873_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
numpy                     1.22.3           py39h18676bf_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
oauthlib                  3.2.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
openh264                  2.1.1                h780b84a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
openjpeg                  2.4.0                hb52868f_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
openssl                   1.1.1n               h166bdaf_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
packaging                 21.3               pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
parso                     0.8.3              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pcre                      8.45                 h9c3ff4c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pickleshare               0.7.5                   py_1003    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pillow                    9.1.0            py39hae2aec6_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pip                       22.0.4             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
prompt-toolkit            3.0.29             pyha770c72_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
protobuf                  3.20.0           py39h5a03fae_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pthread-stubs             0.4               h36c2ea0_1001    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyasn1                    0.4.8                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyasn1-modules            0.2.7                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pycparser                 2.21               pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pygments                  2.11.2             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyjwt                     2.3.0              pyhd8ed1ab_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyparsing                 3.0.7              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyqt                      5.12.3           py39hf3d152e_8    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyqt-impl                 5.12.3           py39hde8b62d_8    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyqt5-sip                 4.19.18          py39he80948d_8    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyqtchart                 5.12             py39h0fcd23e_8    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyqtwebengine             5.12.1           py39h0fcd23e_8    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pysocks                   1.7.1            py39hf3d152e_5    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
python                    3.9.12          h9a8a25e_1_cpython    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
python_abi                3.9                      2_cp39    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pytorch                   1.11.0          py3.9_cuda11.3_cudnn8.2.0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
pytorch-mutex             1.0                        cuda    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
pyu2f                     0.1.5              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyyaml                    5.4.1            py39h3811e60_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pyzmq                     22.3.0           py39headdf64_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
qt                        5.12.9               h1304e3e_6    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
readline                  8.1                  h46c0cb4_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
requests                  2.27.1             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
requests-oauthlib         1.3.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
rsa                       4.8                pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
scikit-video              1.1.11             pyh24bf2e0_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
scipy                     1.8.0            py39hee8e79c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
setuptools                62.0.0           py39hf3d152e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
six                       1.16.0             pyh6c4a22f_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
soupsieve                 2.3.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
sqlite                    3.37.1               h4ff8645_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
stack_data                0.2.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
svt-av1                   0.9.1                h27087fc_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tensorboard               2.8.0              pyhd8ed1ab_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tensorboard-data-server   0.6.0            py39h95dcef6_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tensorboard-plugin-wit    1.8.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
termcolor                 1.1.0                      py_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tk                        8.6.12               h27826a3_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
torchaudio                0.11.0               py39_cu113    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
torchfile                 0.1.0                      py_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
torchvision               0.12.0               py39_cu113    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch
tornado                   6.1              py39hb9d737c_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tqdm                      4.63.0             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
trimesh                   3.10.2             pyh6c4a22f_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
typing-extensions         4.1.1                hd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
typing_extensions         4.1.1              pyha770c72_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tzdata                    2022a                h191b570_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
unicodedata2              14.0.0           py39hb9d737c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
urllib3                   1.26.9             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
visdom                    0.1.8.9                       0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
websocket-client          1.3.2              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
werkzeug                  2.1.1              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
x264                      1!161.3030           h7f98852_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
x265                      3.5                  h924138e_3    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-fixesproto           5.0               h7f98852_1002    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-libx11               1.7.2                h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-libxau               1.0.9                h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-libxext              1.3.4                h7f98852_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xorg-xproto               7.0.31            h7f98852_1007    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
xz                        5.2.5                h516909a_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
yaml                      0.2.5                h7f98852_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
yarl                      1.7.2            py39hb9d737c_2    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
zeromq                    4.3.4                h9c3ff4c_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
zipp                      3.8.0              pyhd8ed1ab_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
zlib                      1.2.11            h166bdaf_1014    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
zstd                      1.5.2                ha95c52a_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
chenhsuanlin commented 2 years ago

This seems weird. Some last things you could perhaps try if you really need to match my results:

Other than these, unfortunately I'm also out of ideas at this point. My best guess is that the discrepancy is probably coming from the subtle randomness accumulated in the long run during the optimization process. It will be very tricky to reproduce the exact same optimization trajectory.

Yannnnnnnnnnnn commented 2 years ago

Thanks for your reply. However, it seems that Pytorch 1.11.0 does not support CUDA 11.6 ( the default version is CUDA 11.3 ). Do you compile Pytorch by yourself?

I will try to set torch.backends.cudnn.deterministic = True and compute the variance of the rotation error.

Yannnnnnnnnnnn commented 2 years ago

I conducted two more experiments with the newest code but got almost the same results.

related model is here: https://drive.google.com/file/d/1a286LwIpobiirUgEgclas6cJtXk0vmHk/view?usp=sharing

fern:flower rotat trans PSNR SSIM LPIPS
flower_2 1.155 0.243 23.68 0.70 0.21
flower_3 1.155 0.243 23.68 0.70 0.21
flower 1.155 0.243 23.68 0.70 0.21
chenhsuanlin commented 2 years ago

The Pytorch I have was from conda with CUDA 11.6 installed. Unfortunately there is really not much else I can do to help at this point, as I have also provided my environment and reproduction. The other thing you could try is to use float64 tensors instead (with double floating point precision) -- sometimes poses are sensitive to these, so the large variance in rotation could be coming from these. Is there a reason you have to reproduce the exact numbers though? As I mentioned earlier, it is very difficult to factor these subtle errors out, and I really would not worry too much about this. I should also re-emphasize that, as mentioned in the paper, the rotation is compared against COLMAP (which is also estimated and not considered ground truth), so it's really not much meaningful trying to lower such rotation error to 0.

Yannnnnnnnnnnn commented 2 years ago

Thank you for being so helpful.

As you said about the results compared to COLMAP, I think I will stop trying to do this unnecessary experiment.

Also, I am sensitive to these numbers because I am familiar with SfM and am more concerned with translations and rotations.

I will close this issue now, and thanks again for your help.

chenhsuanlin commented 2 years ago

No problem! In some sense, this type of numerical instability could be interesting to investigate as well 🙂 Sorry this wasn't of much help in the end!

Yannnnnnnnnnnn commented 2 years ago

Exactly, and that's what I plan to do. The discussion with you has given me new ideas for this topic. Thanks. :smile: