Questions about parametrization

[noisyneighbour]

Hello!

First off, thank you for your work on this project.

I am trying to use SP graph clustering on my own forestry dataset consisting of tree instances and I'm running into difficulties with the instance predictions. In the input my dataset has 2 stuff classes: ground and "other" and one thing class: tree canopies + tree trunks. Each tree is its own instance and they are usually around 20-30m high. In the results however, I am finding that tree trunks are often a separate instance from the tree canopy and the tree canopies are often times split into many instances.

I tried experimenting with various parameters, but mostly achieved similar results. These are some parameters I've experimented with trying to resolve the issue:

pcp_cutoff="[1,5,100],[1,10,100],[1,15,100],[1,20,100]"
pcp_spatial_weight="[1,1e-1,1e-2],[1,1e-2,1e-3]"
instance_k_max=5,10,15
instance_radius=5,10,20,30

Parameters that achieved the best results so far:

```yaml voxel: 0.1 knn: 25 knn_r: 15 knn_step: -1 knn_min_search: 10 ground_threshold: 65 percentile_threshold: 1 ground_scale: 20 pcp_regularization: [0.1, 0.2, 0.3] pcp_spatial_weight: [1e-1, 1e-2, 1e-3] pcp_cutoff: [1, 5, 100] pcp_k_adjacency: 10 pcp_w_adjacency: 1 pcp_iterations: 15 graph_k_min: 1 graph_k_max: 30 graph_gap: [5, 30, 30] graph_se_ratio: 0.3 graph_se_min: 20 graph_cycles: 3 graph_margin: 0.5 graph_chunk: [1e6, 1e5, 1e5] # reduce if CUDA memory errors # Batch construction parameterization sample_segment_ratio: 0.2 sample_segment_by_size: True sample_segment_by_class: False sample_point_min: 32 sample_point_max: 128 sample_graph_r: 50 # set to r<=0 to skip SampleRadiusSubgraphs sample_graph_k: 4 sample_graph_disjoint: True sample_edge_n_min: -1 # [5, 5, 15] sample_edge_n_max: -1 # [10, 15, 25] # Augmentations parameterization pos_jitter: 0.05 tilt_n_rotate_phi: 0.1 tilt_n_rotate_theta: 180 anisotropic_scaling: 0.2 node_feat_jitter: 0 h_edge_feat_jitter: 0 v_edge_feat_jitter: 0 node_feat_drop: 0 h_edge_feat_drop: 0.0 v_edge_feat_drop: 0 node_row_drop: 0 h_edge_row_drop: 0 v_edge_row_drop: 0 drop_to_mean: False ```

Maybe somebody here can help me by pointing out which parameters should I focus my experiments on to resolve this issue?

[drprojects]

It seems your $P_1$ and $P_2$ partitions superpoints may be a bit small. In particular, you may want to try to increase the average number of points per superpoint $\frac{|P_0|}{|P_1|}$. A rule of thumb we tried to follow in our datasets (this is no golden rule) is $\frac{|P_0|}{|P_1|} \sim 30$ and $\frac{|P_1|}{|P_2|} \sim 5$.

To adjust your $P_1$ superpoint sizes, you can play with

• pcp_regularization[0]
• pcp_spatial_weight[0]
• pcp_cutoff[0]

Then, to adjust the panoptic segmentation performance, I would play with instance_radius as you seem to have already tried. This should rule the complexity of the superpoint graph on which the instances are computed. Increasing instance_radius should hopefully ensure that all trunk superpoints are properly connected to the associated canopy superpoints.

If this does not suffice, you may also want to play with the partitioner's parameters too...

Good luck !

PS: if you are working on open forestry data, we would be happy to expand the current codebase with your dataset if you send us a pull request :wink:

[noisyneighbour]

Thank you very much! I'll play around with the parameters and report if it resolves the issue. Unfortunately this specific dataset is not open source, but I'll try to contribute in another way.

On a side note, currently the algorithm expects each instance to have only 1 unique classification. For my use case I would like to train on a dataset that has 2 unique classes in each instance. What do you think would be the best way to get rid of this restriction and allow multiple classes in each instance? I know this goes slightly beyond the definition of panoptic segmentation.

[drprojects]

:thinking: for your multi-class setup, modifying the label manipulation logic all through the project might be quite involved. A simpler workaround would be to convert your label pairs into unique labels and treat these as your labels. Typically I would do something like:

def label_pair_to_pair_label(y1, y2, num_classes_1):
    """
    y1 in ⟦0, num_classes_1⟦
    y2 in ⟦0, num_classes_2⟦

    y in ⟦0, num_classes_1 * num_classes_2⟦
    """
    y = y1 + y2 * num_classes_1
    return y

def pair_label_to_label_pair(y, num_classes_1):
    """
    y in ⟦0, num_classes_1 * num_classes_2⟦

    y1 in ⟦0, num_classes_1⟦
    y2 in ⟦0, num_classes_2⟦
    """
    y1 = y % num_classes_1
    y2 = y // num_classes_1
    return y1, y2

This would allow preserving the current pipeline for labels and you can recover your multi-class labels at loss computation time if need be. Up to you on how you ask your model to predict multiple classes and how you supervise it, though. You would also need to investigate closer if/how this strategy affects the panoptic metrics computation. I cannot provide much more help in this direction, however, since this departs from our supported setting.