Error in evaluation with n_contexts=3

[Shubhendu-Jena]

Hi, thanks for the great work. I ran your DTU evaluation with the view sampler "evaluation_index_dtu_nctx3.json" and it results in the following:

qkv = rearrange(qkv, "(v b) n t -> b n (v t)", v=self.n_frames)

File "/home/shubhendujena/anaconda3/envs/mvsplat/lib/python3.10/site-packages/einops/einops.py", line 591, in rearrange return reduce(tensor, pattern, reduction="rearrange", **axes_lengths) File "/home/shubhendujena/anaconda3/envs/mvsplat/lib/python3.10/site-packages/einops/einops.py", line 533, in reduce raise EinopsError(message + "\n {}".format(e)) einops.EinopsError: Error while processing rearrange-reduction pattern "(v b) n t -> b n (v t)". Input tensor shape: torch.Size([3, 384, 256]). Additional info: {'v': 2}. Shape mismatch, can't divide axis of length 3 in chunks of 2

I'd be grateful if you could help me fix this

Thanks in advance

[donydchen]

Hi @Shubhendu-Jena, thanks for your interest in our work.

May I confirm how did you run the test? I just tested the code, and it works out fine with the following command:

https://github.com/donydchen/mvsplat/blob/b999e4b7f0a94387960d8ec752abe6bb060da6d9/more_commands.sh#L53-L61

It seems that you might forget to use dataset.view_sampler.num_context_views=3. Please double-check that and let me know if you encounter other difficulties. Cheers.

[Shubhendu-Jena]

Hi, sorry for the confusion. Indeed, it works! Thanks for the help. I was also wondering if you had any plans to release the training script for DTU as I was quite keen to fine-tune the model on DTU to see how the performance improves. Thanks again!

[donydchen]

No worries, @Shubhendu-Jena.

However, regarding the DTU training scripts, we don't train on DTU either. As detailed in Fig.5 of the paper, we mainly use DTU to demonstrate the cross-dataset generation capability of MVSplat. Hence, we directly use the RE10K trained model to test on DTU without any additional training/fine-tuning on DTU. The scale of DTU is too small compared to RE10K, so I don't know whether it is warranted to fine-tune DTU.

[Shubhendu-Jena]

Ah, I understand. I wanted to ask, just in case. Thanks for the work again!