Open wanghlsara opened 1 month ago
In order to increase the number of iterations of optimization, I used following code in test_calibration.py
quality_controller = QualityController(capture_volume, charuco)
for _ in range(5):
logger.info("Filtering out worst fitting point estimates")
quality_controller.filter_point_estimates(0.25)
quality_controller.capture_volume.optimize()
and got python exception:
File "C:\WORK\caliscope-main\tests\test_calibration.py", line 140, in test_post_monocalibration
quality_controller.capture_volume.optimize()
File "C:\WORK\caliscope-main\caliscope\calibration\capture_volume\capture_volume.py", line 93, in optimize
self.least_sq_result = least_squares(
File "C:\Github\00.caliscope\.venv\lib\site-packages\scipy\optimize\_lsq\least_squares.py", line 832, in least_squares
f0 = fun_wrapped(x0)
File "C:\Github\00.caliscope\.venv\lib\site-packages\scipy\optimize\_lsq\least_squares.py", line 830, in fun_wrapped
return np.atleast_1d(fun(x, *args, **kwargs))
File "C:\WORK\caliscope-main\caliscope\calibration\capture_volume\capture_volume.py", line 162, in xy_reprojection_error
points_3d = current_param_estimates[capture_volume.point_estimates.n_cameras * CAMERA_PARAM_COUNT:].reshape((capture_volume.point_estimates.n_obj_points, 3)
)
ValueError: cannot reshape array of size 93213 into shape (31065,3)
I've found that after each iteration of optimization, the capture_volume.point_estimates.n_cameras
may become smaller. If I write it as a magic number (equal to the number of videos) in the code, Exception won't happen again.
But it doesn't seem to make sense to modify like this..... So,Looking forward to your suggestions.
Those RMSE are quite large. I will say that as I try to wrap my head around what you are doing I am quite curious about what the actual setup of this is. You have 37 cameras lining a hallway. How are they configured to record synchronized frames?
Just tacking on to this to record some thoughts.... my expectation is that the whole camera array is kinda weakly joined together in the sense that you have many stereopairs that just don't have common views. And you have many common views where they are only linked at the edge of the view where distortion has the greatest impact, and you have lots of disparity in distance to the camera such that you are combining near and far viewpoints which I think would create an interesting tension between trying to minimize the pixel distance error while guessing at the 3D location.
It would take me some time to really get back into the depths of the bundle adjustment and how points are filtered out. I think that what is happening is that you have so many high error 2D points in the bundle adjustment that when they get filtered out they are losing complete points in the 3D point estimates. This is something that is driven by the extremes mentioned in the paragraph above. A resolvable issue, but I'd at least need access to your intermediate data (config.toml and charuco_xy) and it would take some time.
As another shot in the dark, the lines of code I'd suggested you remove in #659 are there to allow use of a double sided board sandwiched between glass with a mirror image board printed on the back. I'd originally created that for the specific use case of cameras that are mounted far from each other on either end of a hallway. It would allow more links between cameras that otherwise couldn't be linked because they can't face the same surface of the board at the same time. So you could try creating one of these mirror boards, uncommenting that code, and running the frame rate at something much less (I feel like 5 fps would be a good starting point).
Those RMSE are quite large. I will say that as I try to wrap my head around what you are doing I am quite curious about what the actual setup of this is. You have 37 cameras lining a hallway. How are they configured to record synchronized frames?
I recorded videos using some industrial cameras with hardware-level frame synchronization capabilities
1.What are the camera resolutions? 1440x1080
2.What is the size and configuration of the board? [charuco] columns = 4 rows = 3 board_height = 35.43 board_width = 44.88 dictionary = "DICT_4X4_100" units = "inch" aruco_scale = 0.75 square_size_overide_cm = 26.6 inverted = false legacy_pattern = false
3.I don't mean to pry, but just out of curiosity, what domain is this in? sports analysis
A resolvable issue, but I'd at least need access to your intermediate data (config.toml and charuco_xy) and it would take some time.
Attachments are the data files, Thanks for your assistance! testdata.zip
※ In addition, in order to get the port pair (8, 10) from xy_charuco.xy, I changed this line
self.pairs = [(i, j) for i, j in combinations(self.ports, 2) if i < j]
to
self.ports.sort()
self.pairs = list(combinations(self.ports, 2))
I mentioned in #659 that I performed extrinsic calibration in a narrow and long rectangular area. I copied some lines related to rmse from calibration.log. Please forgive me that these lines may not be continuous because I ignored the information unrelated to rmse. What confuses me is that the rmse of each port pair looks good, but the overall error is large. What could be the reason? Is it because there are few cameras with overlapping views in the rectangular area? Or is the ratio of FILTERED_FRACTION=0.025 too small? Or is the number of bundle adjustment stages too small? What aspects should I investigate next? I hope to get your advice.