What exactly is done in HAIS/data/stpls3d_inst.py ?

[Ritchizh]

Hi! I have changed the input data and am trying to run train.py. When it comes to trainMerge(), I get the following error:

ValueError: Caught ValueError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "envs/hais/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "envs/hais/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "STPLS3D/HAIS/data/stpls3d_inst.py", line 231, in trainMerge
    instance_label = self.getCroppedInstLabel(instance_label, valid_idxs)
  File "STPLS3D/HAIS/data/stpls3d_inst.py", line 186, in getCroppedInstLabel
    while (j < instance_label.max()):
  File "envs/hais/lib/python3.7/site-packages/numpy/core/_methods.py", line 40, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

Running _stpls3dinst.py line-by-line showed that for some input data files, the function

    def getCroppedInstLabel(instance_label, valid_idxs):
        instance_label = instance_label[valid_idxs]
        j = 0
        while (j < instance_label.max()):
            if (len(np.where(instance_label == j)[0]) == 0):
                instance_label[instance_label == instance_label.max()] = j
            j += 1
        return instance_label

raises an error.

So, could you please clarify: 1) Why do we crop data in _stpls3dinst.py, weren't it already cropped by _prepare_data_inst_instancestpls3d.py ? What are valid_idx returned by crop()? 2) What does getCroppedInstLabel() function do?

UPD.: For some files, crop() function outputs valid_idxs - all False. What can be the problem?

[meidachen]

Hi @Ritchizh

These codes were directly from the official HAIS for processing scannetv2 here. From my understanding, crop() is trying to make sure the number of points does not exceed 'max_npoint' as mentioned by the author of HAIS from the issue you previously opened under HAIS.

getCroppedInstLabel() seems like doing a process to make sure the instance label is starting from 0 and is continues since from crop() it is possible some instances were getting removed and there is a gap in the instance label.

It is hard for me to say what problem you are facing with your data... From my experience when working on HAIS with STPLS3D, the issues were always related to the parameters in the config file or the data itself that contains some quality issue that was not in scannetv2.

[Ritchizh]

Thank you @meidachen your comment really helped: I've understood what crop is doing. So as a quick fix, I am running training now with increased max_npoint parameter. Later I will crop and downsample my pointclouds more accurately than crop() does.

I've also noticed that you had changed voxel size to ~30 cm (2 cm in original HAIS), batch size to 12 (4 in original HAIS), point_aggr_radius, cluster_shift_meanActive - have you grid-searched these parameters, are they optimal for your data?

[meidachen]

I didn't really grid-search these parameters but only compared them with a few other settings. The main thing I found that helped a lot was the data augmentation and adding the class weight to compute the semantic loss. So I can't say they are optimal, but here we only provide a baseline and hope to see better results in the future.

[Ritchizh]

Got it, thank you! :)