search

vye16 / slahmr

MIT License
459 stars 50 forks source link

What is the use of cameras.json? #49

Closed linjiangya closed 8 months ago

linjiangya commented 8 months ago

Dear authors,

Thank you for your great work. I am confused about this cameras.json produced while running run_opt.py.

https://github.com/vye16/slahmr/blob/58518fec991877bc4911e260776589185b828fe9/slahmr/run_opt.py#L63

  1. What is the use of this cameras.json? It differs from all the other camera parameters stored in /smooth_fit or /root_fit or /motion_chunk (e.g. XXX_cameras_000000.json) and it seems not to be used anywhere.
  2. Also, the camera parameters in XXX_cameras_000060.json stored in /smooth_fit/ seem to be different from the camera parameters stored in XXX_000060_world_results.npz. Could you tell me why is the parameters in JSON file needed a transformation like below? (as it's also not used anywhere) image
vye16 commented 8 months ago

Yes, this is primarily an easy debugging output for reading out the initialized cameras in each phase quickly, but it's not used in processing. In smooth fit and motion_chunks, the cameras are updated during the optimization, so they should not match these initial cameras during these phases.

On Mon, Jan 15, 2024 at 5:49 PM linjiangya @.***> wrote:

Dear authors,

Thank you for your great work. I am confused about this cameras.json produced while running run_opt.py.

https://github.com/vye16/slahmr/blob/58518fec991877bc4911e260776589185b828fe9/slahmr/run_opt.py#L63

What is the use of this file? It seems not to be used anywhere.

— Reply to this email directly, view it on GitHub https://github.com/vye16/slahmr/issues/49, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOKWZPUF3ZAF37MLCSUHDYOXMEJAVCNFSM6AAAAABB4BGP6WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4DEOBYGY4TQNA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

linjiangya commented 7 months ago

Yes, this is primarily an easy debugging output for reading out the initialized cameras in each phase quickly, but it's not used in processing. In smooth fit and motion_chunks, the cameras are updated during the optimization, so they should not match these initial cameras during these phases. … On Mon, Jan 15, 2024 at 5:49 PM linjiangya @.> wrote: Dear authors, Thank you for your great work. I am confused about this cameras.json produced while running run_opt.py. https://github.com/vye16/slahmr/blob/58518fec991877bc4911e260776589185b828fe9/slahmr/run_opt.py#L63 What is the use of this file? It seems not to be used anywhere. — Reply to this email directly, view it on GitHub <#49>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOKWZPUF3ZAF37MLCSUHDYOXMEJAVCNFSM6AAAAABB4BGP6WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4DEOBYGY4TQNA . You are receiving this because you are subscribed to this thread.Message ID: @.>

Thank you for your reply. Sorry for reopening this due to some further question

1) I think the default setting in "smooth fit" is opt_cam=False and opt_scale=True. So the camera parameter in smooth_fit will only have different cam_t, while cam_R should keep unchanged right? (I have checked this, and it's true. The cam_t and cam_R are all the same through root_fit folder and the XXX_cameras_000000.json in smooth_fit, and only cam_t will start to change (this is because of the update of world scale) from XXX_cameras_000018.json in smooth_fit folder.

2) I have tested this on a video with static camera but the camera location is visualization is still moving and the camera parameter is still changing over time and shaking sometimes. Is there a way to fix this?

vye16 commented 7 months ago
  1. This is correct.
  2. Perhaps the video is static but the SLAM method computed slightly nonzero camera motion. To deal with this, you can increase the motion threshold in the SLAM preprocessing here. This might cause the SLAM method to fail, but that just means that the cameras are static, and slahmr will use identity cameras.