run on batch data

[whulc]

It seems that dkm_model.match doesn't accept batch input now?

[Parskatt]

I keep changing the format so might have happened. Can you make a simple test case that I can fix?

[whulc]

I want to get the matching result for two 4D tensor(im1, im2) with size b, 3, h, w.

flow, confidence = dkm_model.match(im1, im2, device=device, batched=True)

Here is the error message. /root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None. warnings.warn(msg) 0%| | 0/838 [00:00<?, ?it/s]/root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.) return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined] 0%| | 0/838 [00:03<?, ?it/s] Traceback (most recent call last): File "/mnt/e/3D_tasks/SFM_related/DKM-main/demo_match_opensfm.py", line 238, in match_for_opensfm(args.data_dir, args.width, args.height, args.batch_size) File "/mnt/e/3D_tasks/SFM_related/DKM-main/demo_match_opensfm.py", line 100, in match_for_opensfm flow, confidence = dkm_model.match(im1, im2, device=device, batched=True) File "/mnt/e/3D_tasks/SFM_related/DKM-main/dkm/models/dkm.py", line 707, in match dense_corresps = self.forward_symmetric(batch, upsample = True) File "/mnt/e/3D_tasks/SFM_related/DKM-main/dkm/models/dkm.py", line 638, in forward_symmetric feature_pyramid = self.extract_backbone_features(batch, upsample = upsample) File "/mnt/e/3D_tasks/SFM_related/DKM-main/dkm/models/dkm.py", line 580, in extract_backbone_features feature_pyramid = self.encoder(X) File "/root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, *kwargs) File "/mnt/e/3D_tasks/SFM_related/DKM-main/dkm/models/encoders.py", line 48, in forward x = net.conv1(x) File "/root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 463, in forward return self._conv_forward(input, self.weight, self.bias) File "/root/miniconda3/envs/mmseg/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [2, 2, 3, 540, 960] ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python', '/mnt/e/3D_tasks/SFM_related/DKM-main/demo_match_opensfm.py']' command failed. (See above for error)

Process finished with exit code 1

[Parskatt]

Thanks, I'll look into it.

[Parskatt]

I think the issue is that "upsample_preds" assumes that the data is not batched, since it wants the use the original images for higher resolution. If you set "dkm_model.upsample_preds" to false I think it should work.

[Parskatt]

You still cannot use upsample preds and batched jointly currently. Added assertion for this. With some changes to API I guess it would work. If people are interested enough I could add it.