Closed wanmeihuali closed 1 year ago
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck created
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-train created
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-truck/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-truck/output/output.tar.gz
Training job param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-train completed. Model url: s3://taichi-3d-gaussian-splatting-log/tat-train/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-train/output/model.tar.gz, tensorboard output path: s3://taichi-3d-gaussian-splatting-log/tat-train/param-tune-less-point-318f80a-2023-06-21-21-48-37-tat-train/output/output.tar.gz
truck 30k PSNR: 23.29, SSIM: 0.823, #points 321k, train time(T4): 1h35min train: 30k PSNR: 20.88 SSIM: 0.758, #points 445k, train time(T4): 1h55min
The training time on T4 is much slower than 3090...
Running experiment on sagemaker with git sha 318f80aecbf78709d207298f484fbf1796028d2c