Issue while running inference on custom data

[soumyamanjunath]

Hi We are using Superpoint transform pretrained weights to perform inference on our custom data. We are using Dales pretrained weights to perform the inference on our custom data. Our custom data has other classes . We are mapping the classes in custom data to Dales classes before loading the data.

We are performing this mapping as follows :

DCILDATA_TO_DALESMAP={ 1 : 8.0,
                       2 : 0,
                       3 : 1.0,
                       4 : 1.0 ,
                       5 : 1.0 , 
                       6 : 7.0 , 
                       7:  8.0 ,
                       8 : 8.0 ,
                       9:  8.0 ,
                      10:  8.0 ,
                      11:  8.0 ,
                      12:  8.0 , 
                      13:  8.0 ,
                      14:  8.0 , 
                      15 : 8.0 , 
                      16 : 1.0 ,
                      17:  1.0 ,
                      18:  6.0 , 
                      19:  2.0,
                      20:  4.0 ,
                      21 : 4.0 ,
                      22:  8.0 ,
                      30 : 8.0 ,
                      31: 8.0 ,
                      32 : 8.0 ,
                      33 : 8.0 ,
                      38: 8.0 ,
                      99 : 8.0
                     }

def read_dales_tile(
        filepath, xyz=True, intensity=True, semantic=True, instance=False,
        remap=False):
    data = Data()
    key = 'testing'

    print("~~~~~~~~~~~spt:src:datasets:dales~~~~~~~~~~~~")
    with open(filepath, "rb") as f:
        tile = PlyData.read(f)

        if xyz:
#             print('tile---------------------------------',tile[key][axis])
            data.pos = torch.stack([
                torch.FloatTensor(tile[key][axis])
                for axis in ["x", "y", "z"]], dim=-1)

        if intensity:
            # Heuristic to bring the intensity distribution in [0, 1]
            data.intensity = torch.FloatTensor(
                tile[key]['scalar_Intensity']).clip(min=0, max=60000) / 60000

        **if semantic:
            myclass = tile[key]['scalar_Classification']

            newclass = [DCILDATA_TO_DALESMAP[cls] for cls in myclass]** 
            y = torch.LongTensor(np.array(newclass))
            data.y = torch.from_numpy(ID2TRAINID)[y] if remap else y

        if instance:
            data.instance = torch.LongTensor(tile[key]['ins_class'])
    print("data.pos",data.pos)
    print("data.y",data.y)
    return data

The following error is seen when the code tries to perform transformations in line 147 of superpoint_transformer/src/transforms/sampling.py

RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

Please let us know how can we resolve this issue .

Thanks and Regards Soumya Manjunath

[soumyamanjunath]

Hi , I am also uploading the custom data so that its easier to reproduce the issue . The data is shared here ->https://drive.google.com/drive/folders/1X4WFqqVYnDPjh5N5wUutPf13QpiKWx3h?usp=drive_link This data is in ply ASCII format. I also tried using ply binary format. This data does not have instance information but has intensity and class.

This google drive folder has 2 ply files. The above error _"cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1"_ can be reproduced with testing2.ply

The image attached in this comment shows the another error which can be reproduced with "000101 - Cloud.ply" file _*ValueError: RANSAC could not find a valid consensus set. All max_trials iterations were skipped because each randomly chosen sub-sample failed the passing criteria. See estimator attributes for diagnostics (n_skips).**_

Request you to kindly help us in resolving this issue as soon as possible

Regards Soumya Manjunath

[soumyamanjunath]

Hi, Any update on the above issue

Regards Soumya

[drprojects]

Hi, I am currently out of office, will look into this when I come back, two weeks from now.

[drprojects]

I loaded the testing2.ply data you shared and visualized it. I can already tell there is a problem in your labels mapping.

Following our DALES labeling convention:

• red = building
• eggshell = ground
• blue = unknown

I assume this mapping is not what you want.

If you have a look at the read_dales_tile() code, you will see that we use our own remapping to convert raw DALES labels to our training labels. As explained in the docs (please make sure you read it carefully 🙏), we have a canonical way of organizing semantic labels for all datasets.

If you want to use our pretrained DALES model, your labels must be mapped to our DALES training labels, not the DALES raw labels. Have a look here to see how we remap our DALES labels.

You have two options:

• directly map your labels to our train labels (then you must set remap=False in read_dales_tile()
• map your labels to the official DALES labels (then you must set remap=True in read_dales_tile()

Before looking into anything else, please make sure the output of read_dales_tile() on your own data only produces labels in $[0, 8]$, correctly mapped to our DALES training labels. In particular, points with label $8$ will be treated as void/ignored/unknown/unclassified (whatever you call it). Labels outside the $[0, 8]$ range are likely to cause downstream errors.

[soumyamanjunath]

Hi Thank you for your analysis and detailed reply.

This is custom data on which we would want the model to perform prediction of classes for each point in the point cloud. Does the code implementation filter out certain points with particular labels before providing the data to the model for prediction? Irrespective of the class of every point , the model should be able to provide predictions for each point . Ofcourse , the map and accuracy calculations will get affected if the input labels are wrong. But here our objective is to obtain the predictions on the unseen custom data.

As correctly mentioned by you , "Labels outside the range are likely to cause downstream errors." , This is the reason i remapped the existing labels in the custom data to the Dales format of labeling for ease of execution of inference using pretrained weights.So currently , the custom data has labels in the range 0 to 8

Please let me know how to proceed. Why are we getting the following errors?

Error 1: RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

Error 2: _ValueError: RANSAC could not find a valid consensus set. All max_trials iterations were skipped because each randomly chosen sub-sample failed the passing criteria. See estimator attributes for diagnostics (nskips).*

Best Regards Soumya Manjunath

[drprojects]

Error 1: RuntimeError: cannot reshape tensor of 0 elements into shape [0, -1] because the unspecified dimension size -1 can be any value and is ambiguous

This sounds like your tensor is empty. Please inspect the shape of your coords tensor before calling grid_cluster . If this tensor is indeed empty, or of shape (0, 3), this means your point cloud is empty. You will have to look up upstream how this could happen. Is this because your input point cloud is actually empty ? Make sure the output of read_dales_tile() is a Data object, with at least non-empty pos and intensity attributes (y is optional if you only intend to run inference without computing metrics).

Error 2: ValueError: RANSAC could not find a valid consensus set. All max_trials iterations were skipped because each randomly chosen sub-sample failed the passing criteria. See estimator attributes for diagnostics (n_skips).*

The RANSAC algorithm is used here to search for the ground plane. The message tells you it could not find a plane. Judging from the visualization I made of your testing2.ply, this is fishy. This could be related to the above error: are you sure your Data object is not empty ? Please investigate the shape of your data.

[soumyamanjunath]

The data is not empty I will share more logs asap to identify the issue

[drprojects]

Without further details, I am closing this issue for now.