holes in dense reconstruction

[thunders82]

Hi,

I'm currently trying to reconstruct a head/face but I have few issues. I managed to solve some of them but can't get the bottom of it.

I used 8 high resolution cameras (+/- 8k by 5k) to simultaneously capture the face with flat lighting to avoid shadows and I used polarisers to remove reflections. The generation of the sparse point cloud wasn't working properly with any of the presets. All the features were located is the same areas --> hair and eyebrows. I had to play with the octave number and resolution (40 and 2 respectively) to improve the sparse ptc and recover a better position for the cameras --> mean reprojection error = 0.904433. I believe it is not too bad but what could be considered as a good range value for the mean reprojection error ?

Extra info : I noticed aswell the max_num_feature from the feature extraction is not really respected. Depending on the image the number of features can be double or more the input value. Is it expected ?

From there I really struggle to create a dense ptc without massive holes using any of the presets. Using the full res with the highest of lowest preset doesn't really help. As an attempt to improve the reconstruction I've increased the window radius to 20 which seems to be the maximum allowed ?? and reduce the image size to 2000. This helps a lot but I still have holes around the nose ,nostrils , mouth, eyebrows and eyelashes. I'm not counting the ears as the image coverage is not ideal.

What can I do to improve the reconstruction knowing I reached the maximum window radius and knowing lowering the image size will just cancel the purpose of having high resolution source images.

Thank you!

As I've played with other parameters I copy my project.ini below :

database_path=/pathto/database.db image_path=/pathto/images [ImageReader] single_camera=true single_camera_per_folder=false existing_camera_id=-1 default_focal_length_factor=1.2 mask_path=/pathto/masks camera_model=SIMPLE_RADIAL camera_params= camera_mask_path= [SiftExtraction] use_gpu=true estimate_affine_shape=true upright=false domain_size_pooling=true num_threads=-1 max_image_size=4096 max_num_features=12288 first_octave=-1 num_octaves=40 octave_resolution=2 max_num_orientations=2 dsp_num_scales=15 peak_threshold=0.0066699999999999997 edge_threshold=10 dsp_min_scale=0.086669999999999997 dsp_max_scale=2.5 gpu_index=-1 [SiftMatching] use_gpu=true cross_check=true multiple_models=false guided_matching=true num_threads=-1 max_num_matches=32768 max_num_trials=10000 min_num_inliers=15 max_ratio=0.80000000000000004 max_distance=0.69999999999999996 max_error=4 confidence=0.99990000000000001 min_inlier_ratio=0.25 gpu_index=-1 [SequentialMatching] quadratic_overlap=true loop_detection=false overlap=10 loop_detection_period=10 loop_detection_num_images=50 loop_detection_num_nearest_neighbors=1 loop_detection_num_checks=256 loop_detection_num_images_after_verification=0 loop_detection_max_num_features=-1 vocab_tree_path= [SpatialMatching] is_gps=true ignore_z=true max_num_neighbors=50 max_distance=100 [BundleAdjustment] refine_focal_length=true refine_principal_point=false refine_extra_params=true refine_extrinsics=true max_num_iterations=1000 max_linear_solver_iterations=2000 function_tolerance=1e-100 gradient_tolerance=1e-100 parameter_tolerance=1e-100 [Mapper] ignore_watermarks=false multiple_models=false extract_colors=true ba_refine_focal_length=true ba_refine_principal_point=false ba_refine_extra_params=true ba_global_use_pba=true fix_existing_images=false tri_ignore_two_view_tracks=false min_num_matches=15 max_num_models=50 max_model_overlap=20 min_model_size=10 init_image_id1=-1 init_image_id2=-1 init_num_trials=400 num_threads=-1 ba_local_num_images=6 ba_local_max_num_iterations=40 ba_global_pba_gpu_index=-1 ba_global_images_freq=500 ba_global_points_freq=250000 ba_global_max_num_iterations=100 ba_global_max_refinements=5 ba_local_max_refinements=3 snapshot_images_freq=0 init_min_num_inliers=200 init_max_reg_trials=4 abs_pose_min_num_inliers=30 max_reg_trials=3 tri_max_transitivity=1 tri_complete_max_transitivity=5 tri_re_max_trials=1 min_focal_length_ratio=0.10000000000000001 max_focal_length_ratio=10 max_extra_param=10000000 ba_global_images_ratio=1.1000000000000001 ba_global_points_ratio=1.1000000000000001 ba_global_max_refinement_change=0.00050000000000000001 ba_local_max_refinement_change=0.001 init_max_error=4 init_max_forward_motion=0.94999999999999996 init_min_tri_angle=5 abs_pose_max_error=12 abs_pose_min_inlier_ratio=0.25 filter_max_reproj_error=4 filter_min_tri_angle=1.5 tri_create_max_angle_error=2 tri_continue_max_angle_error=2 tri_merge_max_reproj_error=4 tri_complete_max_reproj_error=4 tri_re_max_angle_error=5 tri_re_min_ratio=0.20000000000000001 tri_min_angle=1.5 snapshot_path= [PatchMatchStereo] geom_consistency=false filter=false write_consistency_graph=false max_image_size=2000 window_radius=20 window_step=2 num_samples=15 num_iterations=5 filter_min_num_consistent=2 depth_min=-1 depth_max=-1 sigma_spatial=-1 sigma_color=0.20000000000000001 ncc_sigma=0.59999999999999998 min_triangulation_angle=1 incident_angle_sigma=0.90000000000000002 geom_consistency_regularizer=0.29999999999999999 geom_consistency_max_cost=3 filter_min_ncc=0.10000000000000001 filter_min_triangulation_angle=3 filter_geom_consistency_max_cost=1 cache_size=32 gpu_index=-1 [Render] adapt_refresh_rate=true image_connections=false min_track_len=0 refresh_rate=1 projection_type=0 max_error=2 [ExhaustiveMatching] block_size=50 [VocabTreeMatching] num_images=100 num_nearest_neighbors=5 num_checks=256 num_images_after_verification=0 max_num_features=-1 vocab_tree_path= match_list_path= [TransitiveMatching] batch_size=1000 num_iterations=3 [StereoFusion] max_image_size=-1 min_num_pixels=5 max_num_pixels=10000 max_traversal_depth=100 check_num_images=50 max_reproj_error=2 max_depth_error=0.01 max_normal_error=10 cache_size=32 [PoissonMeshing] depth=13 num_threads=-1 point_weight=1 color=32 trim=10 [DelaunayMeshing] num_threads=-1 max_proj_dist=20 max_depth_dist=0.050000000000000003 distance_sigma_factor=1 quality_regularization=1 max_side_length_factor=25 max_side_length_percentile=95

[thunders82]

Hi, Any hint/suggestions?

Thanks

[prettybot]

same question. I am doing the similar work. Could you plz share your progress on it? @thunders82 Thanks a lot.

[thunders82]

Hi @prettybot, Adding the metadatas to the images help fixing the issue for me.