Export for Gaussian Splatting - No images saved

[chupinsky27]

I've been struggling with some wide angle data sets not saving out undistorted images and the images.bin file. I am using a DJI Osmo Action 4 on the ultrawide setting. There is considerable distortion yet metashape correctly solves for it along with its rolling shutter. I can even export out undistorted images through File>Export>Convert Images. I have followed the instructions in the script and set the Fixed parameters> "cx, cy" and used Adaptive camera model fitting.

Your help is appreciated.

Here's the log when I run the script:

2024-05-07 17:47:29 All chunks: False 2024-05-07 17:47:29 All frames: False 2024-05-07 17:47:29 Zero cx and cy: True 2024-05-07 17:47:29 Use local coordinate frame: True 2024-05-07 17:47:29 Image quality: 90 2024-05-07 17:47:29 Export images: True 2024-05-07 17:47:29 Confirm deletion: True 2024-05-07 17:47:29 Using pinhole model instead of simple_pinhole: True 2024-05-07 17:47:29 Using only uncropped projections: True 2024-05-07 17:47:32 C:/Users/designer/gaussian-splatting/data/OA4_CarMount_CocaColaStore_01/ 2024-05-07 17:47:32 2024-05-07 17:47:32 C:/Users/designer/gaussian-splatting/data/OA4_CarMount_CocaColaStore_01/ 2024-05-07 17:47:34 Calibrations: 2024-05-07 17:47:34 Camera DJI_20240502183523_0214_D_146821 (key = 0) has cropped sensor (key = 0) ... Repeat (key= 1-125) ... 2024-05-07 17:47:34 Camera DJI_20240502183523_0214_D_146947 (key = 126) has cropped sensor (key = 0) 2024-05-07 17:47:34 Found 0 cropped projections 2024-05-07 17:47:34 Undistorted 0 cameras 2024-05-07 17:47:34 Saved 0 calibrations 2024-05-07 17:47:34 Saved 0 cameras 2024-05-07 17:47:35 Saved 116675 points from 116675 tracks 2024-05-07 17:47:35 2024-05-07 17:47:35 Done

[PolarNick239]

Hi, script exports only Frame cameras because Gaussian Splatting doesn't support any camera type except regular Frame-camera. What Camera type do you have in Tools->Camera Calibration?

Can you please share a screenshot of "Adjusted" tab in Camera Calibration?

[chupinsky27]

Thank you for the reply. I do have the Camera Calibration camera type set to "Frame". Here is a screenshot of the adjusted calibration with "cx, cy" set to fixed:

I also tried doing a precalibration by using the checkerboard pattern and running a Tools>Camera>Calibrate Camera. Even though metashape would properly align the cameras, I could not get the script to save images. Precalibration results here:

to reiterate, I can still successfully export undistorted images through File>Export>Convert Images.

[Teran72]

Hi, we cannot reproduce this problem on our side, so can you, please, send a project-reproducer to support@agisoft.com for further investigation. Part of the dataset should be enough, just be sure that it reproduces the issue.

[chupinsky27]

I just emailed two datasets that reliably reproduce this issue.

[PolarNick239]

Hi, thanks for the provided data, we have reproduced and investigated the problem. In Tools->Camera Calibration...->Right Click on camera->Distortion Plot.... you can see that the error is very large (RMS 464 pixels):

This is most likely due to the fact that your camera/lens cannot be fit with mathematical model good enough. This can be due to non-suitable for photogrammetry lens (f.e. it can be too complicated to be modeled) or due to dataset specifics.

In this dataset I tried to Tools->Optimize Cameras with all calibration parameters enabled for fitting, but the error is still too large (switching camera to Fisheye doesn't help either). So probably this is due to the pattern of dataset photos - they form straight line (no side overlap => so no vertical parallax) + there are insufficient data at the top and bottom of the photos. May be there are also some kind of undistortion or stabilization in the camera - try to disabled them, because when they are enabled - image is changed and optical geometry cannot be modeled with standard mathematical model.

I can recommend you to solve this problem with large calibration error first. You can try to pre-calibrate camera with many photos (50-100) of highly textured object (any two photos overlapped, and all pixels of each photo observes highly textured surface) and ensure that this project with highly textured object has low calibration distortion error. And try to disable all images post-processing in camera (stabilizer/undistortion).

[PolarNick239]

script exports only Frame cameras because Gaussian Splatting doesn't support any camera type except regular Frame-camera. What Camera type do you have in Tools->Camera Calibration?

Hi! FYI - fisheye camera type is now supported - see https://github.com/agisoft-llc/metashape-scripts/issues/89#issuecomment-2244826331