Issue in code?

[yohanshin]

Hi,

I am running SLAHMR on some long-sequence video and I realize that there seems to have small mistake in your code.

In dataset.py line:325-327, I think you intended to normalize camera translation to the origin when we are splitting the sequence into small batches. However, since t0 is defined as:

t0 = -cam_t[sidx:sidx+1] + torch.randn(3) * 0.1

I believe the code at line:327 needs to be changed to: self.cam_t = cam_t[sidx:eidx] - t0 --> self.cam_t = cam_t[sidx:eidx] + t0 .

When I fix this, it works well on long-range video. Let me know if this is correct way to fix the code.

[HenrySomeCode]

Hi,

I am running SLAHMR on some long-sequence video and I realize that there seems to have small mistake in your code.

In dataset.py line:325-327, I think you intended to normalize camera translation to the origin when we are splitting the sequence into small batches. However, since t0 is defined as:

t0 = -cam_t[sidx:sidx+1] + torch.randn(3) * 0.1

I believe the code at line:327 needs to be changed to: self.cam_t = cam_t[sidx:eidx] - t0 --> self.cam_t = cam_t[sidx:eidx] + t0 .

When I fix this, it works well on long-range video. Let me know if this is correct way to fix the code.

Hi, in my case, If I change the line 327 in dataset.py from self.cam_t = cam_t[sidx:eidx] - t0 to self.cam_t = cam_t[sidx:eidx] + t0. I get this error:

[Errno 2] No such file or directory: 
'.../slahmr/slahmr/inputs/k7_2/slahmr/phalp_out/phalp_out/k7/results/demo_k7.pkl' -> 
'.../slahmr/slahmr/inputs/k7_2/slahmr/phalp_out/phalp_out/k7/k7.pkl'

The command I run is python run_opt.py data=video run_opt=False run_vis=False. In normal use, when I remain the line 327 as self.cam_t = cam_t[sidx:eidx] - t0, I get this error: IndexError: index 0 is out of bounds for axis 0 with size 0 Then I changed these lines in video.yaml from

tracks: ${data.root}/slahmr/track_preds/${data.seq}
shots: ${data.root}/slahmr/shot_idcs/${data.seq}.json

to

tracks: ${data.root}/slahmr/phalp_out/track_preds/${data.seq}
shots: ${data.root}/slahmr/phalp_out/shot_idcs/${data.seq}.jsont

Then run the command `python run_opt.py data=video run_opt=True run_vis=True. After that, the code outputs the output video. I think that I might being wrong at some points. Could you please show me that?

[geopavlakos]

@yohanshin, could you share an example of a longer video that run successfully with this change but not with the original code? Thank you!

[yohanshin]

Hi @geopavlakos, sincere apologize for the late response.

Indeed, even by changing the code to that, I could not get results as good as cutting into chunks. I think I coincidently got some good examples from one random sequence of the EMDB dataset. So once I realize this, I stopped running SLAHMR on the entire frames, and followed your recommended setting (i.e., crop videos to 100 frames chunks and run each chunk individually).