Illegal pointer access | memory corruption thrown | segmentation error thrown while updating esdf using depth frame or querying sdf query

[mahajanparth]

Hey i am getting segmentation fault or illegal memory pointer access while running a custom dummy example with nvblox_torch

CUDA : cuda_12.1.r12.1/compiler.32415258_0 GPU : 3070 ti OS : Ubuntu

.

Can somebody tell me what could be the issue here. I have attached segmentation cygdb screenshot for perusal.

Link to download .npy file used in the code : https://zenodo.org/records/7130512/files/mpinets_real_point_cloud_data.npy?download=1

The code i was experimenting it on : import numpy as np

import open3d as o3d

import cv2 import torch import time import examples.demo_utils as demo_utils from nvblox_torch.mapper import (Mapper,) # create_nvblox_mapper_instance, get_nvblox_mapper_class from transforms3d import affines, euler from examples.timer import Timer import faulthandler import time faulthandler.enable()

def get_default_pose():

eul = [-2.2, 0.08750253722127824, 1.4509891417591223]

# p = [ 6, 0,  4]
eul = [-2.0, 0, 0]
p = [1.0, -1.5, 0.5]
R = euler.euler2mat(*eul)
T = affines.compose(p, R, [1, 1, 1])
return np.array(T)

def main():

timer = Timer()
data= np.load("/home/parth/Downloads/mpinets_real_point_cloud_data.npy",allow_pickle=True)
mapper = Mapper([0.01], ["tsdf"])
mapper.clear()
data= data.item()
#print(data)
intrinsic=data["intrinsics"]
depth_img=data["external_xyz_image"][:,:,2]
depth_img[depth_img > 6] = 0.0 
color_img=data["rgb_image"][1]
color_img=np.dstack([color_img, np.ones(shape=(color_img.shape[0],color_img.shape[1]))*255]).astype("uint8")
cv2.imshow("color", color_img)
cv2.imshow("depth", depth_img)
cv2.waitKey(0)

pose=get_default_pose()
c_img= torch.from_numpy(color_img)
d_img= torch.from_numpy(depth_img).float()
pose=torch.from_numpy(pose).float()
intrinsic= torch.from_numpy(intrinsic).float()
print(d_img.shape)
print(c_img.shape)
rgba, depth,pose,intrinsic=demo_utils.prep_inputs(c_img,d_img,pose,intrinsic)
print(pose,intrinsic)

**mapper.add_depth_frame(depth, pose, intrinsic, mapper_id=0)** ( Error thrown here)
#mapper.add_color_frame(rgba, pose, intrinsic, mapper_id=0)

mapper.update_esdf(0)
time.sleep(1)
for i in range(10):
    start_time=time.time()
    batch_size = 600000
    tensor_args = {"device": "cuda", "dtype": torch.float32}
    query_spheres = torch.rand((batch_size, 4), **tensor_args)* 5
    query_spheres[:, 3] = 0.001
    out_points = torch.zeros((batch_size, 4), **tensor_args) + 0.0
    print("QUERYING SDF\n")
    r = mapper.query_sdf(query_spheres, out_points, True, 0)
    r=r.detach().cpu().numpy()

    print("R",r.shape)
    print("UNIQUES",np.unique(r).shape)
    print("End time : ",time.time()-start_time)

demo_utils.show_renders(mapper, pose, "current_pose", timer, mapper_id=0)
cv2.waitKey(0)
cv2.destroyAllWindows()
return None 

if name== "main": main()

[balakumar-s]

We have a fix to make nvblox more stable when running with pytorch. This will be pushed today. In the mean time, you can try to use nvblox_torch through a pytorch docker.

[mahajanparth]

Thank you for the suggestion will try out the same !

[mahajanparth]

Hey I am still facing some issues while using pytorch docker. May i know when can we expect the fix or push ?

[balakumar-s]

We pushed a new release that should fix the issue. Can you try with the new install instructions and let us know?

[mahajanparth]

After Following the new instruction , Working robustly :) Thank you so much !