LieTorch Error: Cannot reshape tensor of 0 elements into shape ... When running TartanAir Evaluation

[MarkChenYutian]

Hi, I'm installing DPVO using the provided docker image with CUDA version of 11.7.

When I'm running the evaluation for TartanAir dataset, the following exception is raised:

(dpvo) root@docker-desktop:/DPVO# python evaluate_tartan.py --trials=5 --split=validation --plot --save_trajectory
Running with config...
BUFFER_SIZE: 2048
GRADIENT_BIAS: False
KEYFRAME_INDEX: 4
KEYFRAME_THRESH: 15.0
MIXED_PRECISION: True
MOTION_DAMPING: 0.5
MOTION_MODEL: DAMPED_LINEAR
OPTIMIZATION_WINDOW: 10
PATCHES_PER_FRAME: 96
PATCH_LIFETIME: 13
REMOVAL_WINDOW: 22
Traceback (most recent call last):
  File "/DPVO/evaluate_tartan.py", line 182, in <module>
    results = evaluate(cfg, args.weights, split=args.split, trials=args.trials, plot=args.plot, save=args.save_trajectory)
  File "/root/miniconda3/envs/dpvo/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/DPVO/evaluate_tartan.py", line 110, in evaluate
    traj_est, tstamps = run(scene_path, config, net)
  File "/root/miniconda3/envs/dpvo/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/DPVO/evaluate_tartan.py", line 54, in run
    slam.update()
  File "/DPVO/dpvo/dpvo.py", line 272, in update
    coords = self.reproject()
  File "/DPVO/dpvo/dpvo.py", line 188, in reproject
    coords = pops.transform(SE3(self.poses), self.patches, self.intrinsics, ii, jj, kk)
  File "/DPVO/dpvo/projective_ops.py", line 60, in transform
    Gij = poses[:, jj] * poses[:, ii].inv()
  File "/DPVO/dpvo/lietorch/groups.py", line 147, in inv
    return self.__class__(self.apply_op(Inv, self.data))
  File "/DPVO/dpvo/lietorch/groups.py", line 130, in apply_op
    return data.view(out_shape + (-1,))
RuntimeError: cannot reshape tensor of 0 elements into shape [1, 0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

Does anyone know how to solve this problem?

---

More Environment Information

``` $ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed_Jun__8_16:49:14_PDT_2022 Cuda compilation tools, release 11.7, V11.7.99 Build cuda_11.7.r11.7/compiler.31442593_0 ``` ``` $ nvidia-smi +-----------------------------------------------------------------------------+ | NVIDIA-SMI 515.57 Driver Version: 516.59 CUDA Version: 11.7 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A | | N/A 50C P5 25W / N/A | 1289MiB / 6144MiB | 7% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 20 G /Xwayland N/A | | 0 N/A N/A 22 G /Xwayland N/A | | 0 N/A N/A 33 G /Xwayland N/A | +-----------------------------------------------------------------------------+ ``` ``` $ python --version Python 3.10.0 ```

[MarkChenYutian]

Resolved.

To run TartanAir evaluation, the dataset should be in path datasets/mono or datasets/TartanAir.

When the video iterator failed to read content, it will fail silently and no exception will be raised.

The error only show up in the update phase, since no image is fed into slam, the patch graph is a null graph and has no edge to optimize / reproject / etc.

adding a line to detect whether the video iterator (in evaluate_tartan.py) reads successfully will help a lot to prevent this kind of problem ...

def video_iterator(imagedir, ext=".png", preload=True):
    imfiles = glob.glob(osp.join(imagedir, "*{}".format(ext)))

    data_list = []
    for imfile in sorted(imfiles)[::STRIDE]:
        image = torch.from_numpy(cv2.imread(imfile)).permute(2,0,1)
        intrinsics = torch.as_tensor([fx, fy, cx, cy])
        data_list.append((image, intrinsics))

    if len(data_list) == 0: raise Exception("Failed to read input.")

    for (image, intrinsics) in data_list:
        yield image.cuda(), intrinsics.cuda()