RGB-D Input

[felipesce]

Hey there!

I'm trying to use DROIDSLAM with RGBD input.

I noticed that the track function in droid.py accepts depth as an input, but no info on the correct format, or if something more must be done.

This is my new image_stream function: `def image_stream(imagedir, depthdir, calib, stride): calib = np.loadtxt(calib, delimiter=" ") fx, fy, cx, cy = calib[:4]

K = np.eye(3)
K[0,0] = fx
K[0,2] = cx
K[1,1] = fy
K[1,2] = cy

image_list = sorted(os.listdir(imagedir))[::stride]
depth_list = sorted(os.listdir(depthdir))[::stride]

for t, (imfile,dfile) in enumerate(zip(image_list,depth_list)):
    image = cv2.imread(os.path.join(imagedir, imfile))
    depth = cv2.imread(os.path.join(depthdir, dfile),-1)

    if len(calib) > 4:
        image = cv2.undistort(image, K, calib[4:])

    h0, w0, _ = image.shape
    h1 = int(h0 * np.sqrt((384 * 512) / (h0 * w0)))
    w1 = int(w0 * np.sqrt((384 * 512) / (h0 * w0)))

    image = cv2.resize(image, (w1, h1))
    image = image[:h1-h1%8, :w1-w1%8]
    image = torch.as_tensor(image).permute(2, 0, 1)

    depth = cv2.resize(depth, (w1, h1))
    depth = depth[:h1-h1%8, :w1-w1%8]
    depth = torch.as_tensor(depth)

    intrinsics = torch.as_tensor([fx, fy, cx, cy])
    intrinsics[0::2] *= (w1 / w0)
    intrinsics[1::2] *= (h1 / h0)

    yield t, image[None], depth[None], intrinsics`

This is my current error:

RuntimeError: expand(torch.FloatTensor{[0, 41, 584]}, size=[41, 73]): the number of sizes provided (2) must be greater or equal to the number of dimensions in the tensor (3)

Any help on this matter would be greatly appreciated, thanks for this amazing software.

[Javier12]

Same issue here

[xiehousen]

Try this :

        depth = torch.as_tensor(depth)
        depth = F.interpolate(depth[None,None], (h1, w1)).squeeze()
        depth = depth[:h1-h1%8, :w1-w1%8]
         yield t, image[None], depth, intrinsics

[felipesce]

Thank you! May the gods bless you with a happy and fulfilling life!

[zovelsanj]

Try this :

        depth = torch.as_tensor(depth)
        depth = F.interpolate(depth[None,None], (h1, w1)).squeeze()
        depth = depth[:h1-h1%8, :w1-w1%8]
         yield t, image[None], depth, intrinsics

I did the exact same thing but during the reconstruction process empty point clouds are generated. Do i need to make changes in other parts of the source code?