How is the semantic mask generated?

[centiLinda]

Hi author, thanks for sharing your work!

My 1st question is, for the semantic masks (oracle.png for ETH-UCY, xxx_mask.png for SDD), are they manually labelled or generated by a segmentation network?

In Y-Net, the author uses manually labelled 5-class semantic image to fine-tune the pretrained segmentation network, and uses the result from the segmentation network for training and testing. This way they avoid using ground-truth semantic label during testing (since the semantic image is generated from a trained network).

My 2nd question is, in your supplementary material you mentioned that you use 2 classes for ETH-UCY, 3 classes for SDD. But when I check the SDD masks, they have more than 3 classes:

As shown above, bookstore_0 has 5 classes and coupa_3 has 4 classes. Did you merge them into 3 classes somewhere in your code?

Thank you!

[centiLinda]

I have solved the 2nd question already. In function environment() https://github.com/realcrane/Human-Trajectory-Prediction-via-Neural-Social-Physics/blob/26de46574326d095ae26de9d45a23e8da0b34aa8/model_nsp_wo.py#L12-L220 Semantic class 3&5 means unwalkable, class 4 means weakly-repulsive. The rest of the classes mean walkable.

[centiLinda]

For the 1st question, I checked Y-net's repo and it seems like the semantic masks are the manually labelled masks from Y-net. Is it correct?