the demo_registration is low RAM efficient and low gpu memory efficient

[lfxx]

Thanks for your excellent work first,i get some problems here. 1.Run i run demo_registration.py with kitti like pcd files,use fcgf i can get a large feature which shape is [45793,3]。 When run the code below to calculate corr_pos,the process will be killed due to a huge memory usage which up to 16GB!

# distance = np.sqrt(2 - 2 * (src_features @ tgt_features.T) + 1e-6)
# source_idx = np.argmin(distance, axis=1)
# source_dis = np.min(distance, axis=1)
# corr = np.concatenate([np.arange(source_idx.shape[0])[:, None], source_idx[:, None]], axis=-1)
# src_keypts = src_pts[corr[:,0]]
# tgt_keypts = tgt_pts[corr[:,1]]
# corr_pos = np.concatenate([src_keypts, tgt_keypts], axis=-1)
# corr_pos = corr_pos - corr_pos.mean(0)

This problems has been solved by using fcgf_feature_matching function which is borrowed from DGR instead of the code above. 2.Then i downsample the pcd,now the feature shape is [25000,3],when start to run the code below,the process is been killed again due to a huge gpu memory use which up to 20GB!

res = model(data)

I have register the feature which has shape [45793,3] successfully using DGR with only 8GB gpu memory usage,which is quite fast.is that normal this repo could use 20GB gpu memory?I need your help to solve the problems above!

[XuyangBai]

Hi, thanks for your interest in our work.

For the first question, yes the way I compute the correspondences given the initial features is not memory efficient. You could use KDTree instead for the nearest neighbor search.

from scipy.spatial import cKDTree
feat1tree = cKDTree(tgt_features)
dists, nn_inds = feat1tree.query(src_features , k=1, n_jobs=-1)

For the GPU memory, unfortunately, PointDSC uses a nonlocal operator in the feature extraction part, where the computation complexity is quadratic to the input length (number of correspondences). And that part is necessary since we are trying to explore the pairwise spatial relationship between correspondences. I suggest using mutual check to reduce the size of input correspondences if the input point cloud is quite large.

[lfxx]

Hi, thanks for your interest in our work.

For the first question, yes the way I compute the correspondences given the initial features is not memory efficient. You could use KDTree instead for the nearest neighbor search.

from scipy.spatial import cKDTree
feat1tree = cKDTree(tgt_features)
dists, nn_inds = feat1tree.query(src_features , k=1, n_jobs=-1)

For the GPU memory, unfortunately, PointDSC uses a nonlocal operator in the feature extraction part, where the computation complexity is quadratic to the input length (number of correspondences). And that part is necessary since we are trying to explore the pairwise spatial relationship between correspondences. I suggest using mutual check to reduce the size of input correspondences if the input point cloud is quite large.

Thanks for your kindly reply!Could you give a demo about mutual check since i am a newbee at pointcloud registration.

[XuyangBai]

You can see the following code,

https://github.com/XuyangBai/PointDSC/blob/b009d536ac10b570853833f2178397c154745da9/datasets/ThreeDMatch.py#L110-L119

Basically, you need to first compute the nn_idx from source to target (denoted as source_idx) and nn_idx from target to source (denoted as target_idx) and if target_idx[source_idx[i]] == i then the i th correspondences will be the mutual nearest. mutual_nearest = (target_idx[source_idx] == np.arange(source_idx.shape[0])) can help you find the mutual nearest neighbor. You can replace my code to compute the nn_idx by using KDTree for memory effecicy.

[lfxx]

You can see the following code,

https://github.com/XuyangBai/PointDSC/blob/b009d536ac10b570853833f2178397c154745da9/datasets/ThreeDMatch.py#L110-L119

Basically, you need to first compute the nn_idx from source to target (denoted as source_idx) and nn_idx from target to source (denoted as target_idx) and if target_idx[source_idx[i]] == i then the i th correspondences will be the mutual nearest. mutual_nearest = (target_idx[source_idx] == np.arange(source_idx.shape[0])) can help you find the mutual nearest neighbor. You can replace my code to compute the nn_idx by using KDTree for memory effecicy.

Using Kdtree can improve memory effecicy indeed.But it is too slow.Use the feature which has shape [45793,3] can cost 28s on i7-7700K. The below code is borrowed from DGR:

def fcgf_feature_matching(feats0, feats1):
    '''
    Step 2: coarsely match FCGF features to generate initial correspondences
    '''
    nns = find_knn_gpu(feats0,
                       feats1,
                       nn_max_n=250,
                       knn=1,
                       return_distance=False)
    corres_idx0 = torch.arange(len(nns)).long().squeeze()
    corres_idx1 = nns.long().squeeze()

    return corres_idx0, corres_idx1

This code is very fast to compute corr.But i don't know how to add mutual check into it.Could you help me out.Thank you very much!

[lfxx]

You can see the following code, https://github.com/XuyangBai/PointDSC/blob/b009d536ac10b570853833f2178397c154745da9/datasets/ThreeDMatch.py#L110-L119

Basically, you need to first compute the nn_idx from source to target (denoted as source_idx) and nn_idx from target to source (denoted as target_idx) and if target_idx[source_idx[i]] == i then the i th correspondences will be the mutual nearest. mutual_nearest = (target_idx[source_idx] == np.arange(source_idx.shape[0])) can help you find the mutual nearest neighbor. You can replace my code to compute the nn_idx by using KDTree for memory effecicy.

Using Kdtree can improve memory effecicy indeed.But it is too slow.Use the feature which has shape [45793,3] can cost 28s on i7-7700K. The below code is borrowed from DGR:

def fcgf_feature_matching(feats0, feats1):
    '''
    Step 2: coarsely match FCGF features to generate initial correspondences
    '''
    nns = find_knn_gpu(feats0,
                       feats1,
                       nn_max_n=250,
                       knn=1,
                       return_distance=False)
    corres_idx0 = torch.arange(len(nns)).long().squeeze()
    corres_idx1 = nns.long().squeeze()

    return corres_idx0, corres_idx1

This code is very fast to compute corr.But i don't know how to add mutual check into it.Could you help me out.Thank you very much!

I have resolve this issue.Thanks again!