search

wanmeihuali / taichi_3d_gaussian_splatting

An unofficial implementation of paper 3D Gaussian Splatting for Real-Time Radiance Field Rendering by taichi lang.
Apache License 2.0
648 stars 61 forks source link

Enable camera pose optimization #83

Closed wanmeihuali closed 11 months ago

github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 709bdb05d4f2500180302416e1e70bdff96cebbe

github-actions[bot] commented 1 year ago

Training job grad-pose-709bdb0-230704-090041-tat-truck created

github-actions[bot] commented 1 year ago

Training job grad-pose-709bdb0-230704-090041-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-709bdb0-230704-090041-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-709bdb0-230704-090041-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-709bdb0-230704-090041-tat-truck final metrics:

Latest Metrics

train:loss train:num_valid_points val:ssim val:psnr train:ssimloss train:iteration train:ssim train:psnr val:loss train:l1loss
0.06078709661960602 439065.0 0.8400789499282837 24.494356155395508 0.15725672245025635 30000.0 0.8427432775497437 23.737594604492188 0.06027264520525932 0.036669690161943436

Max Metrics

val:7kpsnr train:5kpsnr val:5kssim train:7kpsnr val:ssim train:5kssim val:psnr val:7kssim train:ssim train:psnr val:5kpsnr train:7kssim
23.05255699157715 23.579914093017578 0.7748168706893921 23.98195457458496 0.8400789499282837 0.8155559301376343 24.494356155395508 0.802121102809906 0.8427432775497437 23.737594604492188 22.16150665283203 0.8366769552230835
github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 1df60520bb2861e7bcfb7fd27426b1289e586c4a

github-actions[bot] commented 1 year ago

Training job grad-pose-1df6052-230705-231308-tat-truck created

github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 0208816f5dc3ed1eb7ff78edcef288b0427a2776

github-actions[bot] commented 1 year ago

Training job grad-pose-0208816-230706-002435-tat-truck created

github-actions[bot] commented 1 year ago

Training job grad-pose-1df6052-230705-231308-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-1df6052-230705-231308-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-1df6052-230705-231308-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-1df6052-230705-231308-tat-truck final metrics:

Latest Metrics

train:psnr val:ssim val:psnr train:l1loss train:ssim train:iteration train:loss val:loss train:num_valid_points train:ssimloss
24.532657623291016 0.8387011289596558 24.460512161254883 0.0327344574034214 0.826497495174408 30000.0 0.06088806688785553 0.060897521674633026 432029.0 0.17350250482559204

Max Metrics

val:7kssim train:psnr val:psnr val:ssim val:5kssim train:ssim train:7kssim val:7kpsnr train:5kpsnr train:7kpsnr train:5kssim val:5kpsnr
0.8007435202598572 24.532657623291016 24.460512161254883 0.8387011289596558 0.7761763334274292 0.826497495174408 0.796715259552002 22.98880386352539 21.92499542236328 23.079547882080078 0.7811781764030457 22.268699645996094
github-actions[bot] commented 1 year ago

Training job grad-pose-0208816-230706-002435-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-0208816-230706-002435-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-0208816-230706-002435-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-0208816-230706-002435-tat-truck final metrics:

Latest Metrics

train:num_valid_points train:iteration train:psnr train:loss val:loss val:ssim train:l1loss train:ssimloss val:psnr train:ssim
441032.0 30000.0 25.386234283447266 0.04868044704198837 0.06080722063779831 0.8391211628913879 0.0295904278755188 0.12504053115844727 24.44965171813965 0.8749594688415527

Max Metrics

val:5kssim val:7kpsnr train:5kssim train:psnr train:7kpsnr train:7kssim val:ssim val:7kssim train:5kpsnr val:psnr train:ssim val:5kpsnr
0.7760491967201233 23.110506057739258 0.7793401479721069 25.386234283447266 22.138338088989258 0.7807836532592773 0.8391211628913879 0.8031792640686035 22.830810546875 24.44965171813965 0.8749594688415527 22.251619338989258
github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 57791f79ce697ad9e7fb43c8ac1e6df8831749b2

github-actions[bot] commented 1 year ago

Training job grad-pose-57791f7-230707-044608-tat-truck created

github-actions[bot] commented 1 year ago

Training job grad-pose-57791f7-230707-044608-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-57791f7-230707-044608-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-57791f7-230707-044608-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-57791f7-230707-044608-tat-truck final metrics:

Latest Metrics

train:iteration train:l1loss val:loss train:loss train:num_valid_points train:ssimloss train:psnr train:ssim val:ssim val:psnr
30000.0 0.03323899954557419 0.06086798757314682 0.061937350779771805 432880.0 0.17673075199127197 24.97323226928711 0.823269248008728 0.839034914970398 24.441146850585938

Max Metrics

val:7kssim train:7kssim val:7kpsnr train:5kpsnr val:5kpsnr train:5kssim train:7kpsnr train:psnr train:ssim val:ssim val:5kssim val:psnr
0.8007597327232361 0.8006725311279297 22.96923828125 24.135854721069336 22.243276596069336 0.8431853652000427 22.518848419189453 24.97323226928711 0.823269248008728 0.839034914970398 0.7759909629821777 24.441146850585938
wanmeihuali commented 1 year ago

Figure_1 Although the NeRF training seems does not improve with pose optimization, the grad for camera pose shall work. The script in pose_estimation.py optimizes the camera pose for a trained scene, and it can refine the camera pose with noise. If the pose grad does not make anything worse, we can try merge it first.

github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 0111719df8fdc48764ce44cc53f40bcd7ebd9c23

github-actions[bot] commented 1 year ago

Training job grad-pose-0111719-230709-062247-tat-truck created

github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha a2309cdf47c91cc4b9b5c4f7018f76b9c292f4fe

github-actions[bot] commented 1 year ago

Training job grad-pose-a2309cd-230709-063055-tat-truck created

github-actions[bot] commented 1 year ago

Training job grad-pose-0111719-230709-062247-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-0111719-230709-062247-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-0111719-230709-062247-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-0111719-230709-062247-tat-truck final metrics:

Latest Metrics

val:ssim train:l1loss train:num_valid_points train:iteration val:loss train:loss train:ssimloss train:ssim val:psnr train:psnr
0.4716382622718811 0.15424515306949615 166307.0 30000.0 0.23722197115421295 0.23633933067321777 0.5647160410881042 0.43528395891189575 13.317609786987305 13.638601303100586

Max Metrics

train:7kssim val:ssim val:7kpsnr val:5kssim train:5kssim val:5kpsnr train:5kpsnr val:psnr train:7kpsnr train:ssim val:7kssim train:psnr
0.47828158736228943 0.4716382622718811 17.74064064025879 0.7760130167007446 0.8111699819564819 22.261547088623047 23.88247299194336 13.317609786987305 17.27647590637207 0.43528395891189575 0.5434229969978333 13.638601303100586
github-actions[bot] commented 1 year ago

Training job grad-pose-a2309cd-230709-063055-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-a2309cd-230709-063055-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-a2309cd-230709-063055-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-a2309cd-230709-063055-tat-truck final metrics:

Latest Metrics

train:psnr train:ssim val:psnr val:loss train:loss train:iteration val:ssim train:l1loss train:num_valid_points train:ssimloss
15.990808486938477 0.4629014730453491 12.337618827819824 0.2595042884349823 0.19545842707157135 30000.0 0.45191073417663574 0.11004839837551117 323516.0 0.5370985269546509

Max Metrics

train:psnr train:ssim val:psnr train:7kssim val:7kpsnr train:7kpsnr val:ssim val:5kpsnr train:5kpsnr val:5kssim val:7kssim train:5kssim
15.990808486938477 0.4629014730453491 12.337618827819824 0.4875752925872803 14.27987289428711 15.284371376037598 0.45191073417663574 22.11170768737793 22.80233383178711 0.772071361541748 0.47993865609169006 0.7824743986129761
github-actions[bot] commented 1 year ago

Running experiment on sagemaker with git sha 81541884575489e397885796a274e78c891e6023

github-actions[bot] commented 1 year ago

Training job grad-pose-8154188-230709-175345-tat-truck created

github-actions[bot] commented 1 year ago

Training job grad-pose-8154188-230709-175345-tat-truck completed.

Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-8154188-230709-175345-tat-truck/output/model.tar.gz,

tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/grad-pose-8154188-230709-175345-tat-truck/output/output.tar.gz

github-actions[bot] commented 1 year ago

Training job grad-pose-8154188-230709-175345-tat-truck final metrics:

Latest Metrics

train:psnr train:ssim train:loss train:l1loss val:loss val:ssim train:ssimloss train:iteration val:psnr train:num_valid_points
24.723766326904297 0.8404524326324463 0.056935884058475494 0.03128296136856079 0.060437630861997604 0.8397347331047058 0.1595475673675537 30000.0 24.469940185546875 432427.0

Max Metrics

train:psnr train:ssim train:5kssim val:5kpsnr train:5kpsnr train:7kpsnr val:ssim train:7kssim val:7kssim val:5kssim val:7kpsnr val:psnr
24.723766326904297 0.8404524326324463 0.7363717555999756 22.2385311126709 21.535194396972656 23.574939727783203 0.8397347331047058 0.8256381750106812 0.8007689714431763 0.7736835479736328 23.058002471923828 24.469940185546875