Questions about Loss

[GuoxingYan]

Hi @zorzi-s, much appreciated for sharing your interesting and knowledgeable work! I have a few questions I'd like to ask you.

1. When calculating the matched loss function, whether the true value needs to be in a one-to-one correspondence with the predicted value. Indicates whether the dustbin in the superglue is used. Check whether the training loss can be obtained by referring to the following code: ` dists = cdist(kp1_projected, kp2_np)

   min1 = np.argmin(dists, axis=0) 
   min2 = np.argmin(dists, axis=1)
   
   min1v = np.min(dists, axis=1) 
   min1f = min2[min1v < 3] 
   
   xx = np.where(min2[min1] == np.arange(min1.shape[0]))[0]
   matches = np.intersect1d(min1f, xx) 
   
   missing1 = np.setdiff1d(np.arange(kp1_np.shape[0]), min1[matches])
   missing2 = np.setdiff1d(np.arange(kp2_np.shape[0]), matches)
   
   MN = np.concatenate([min1[matches][np.newaxis, :], matches[np.newaxis, :]])
   MN2 = np.concatenate([missing1[np.newaxis, :], (len(kp2)) * np.ones((1, len(missing1)), dtype=np.int64)])
   MN3 = np.concatenate([(len(kp1)) * np.ones((1, len(missing2)), dtype=np.int64), missing2[np.newaxis, :]])
   all_matches = np.concatenate([MN, MN2, MN3], axis=1)`

loss = [] for i in range(len(all_matches[0])): x = all_matches[0][i][0] y = all_matches[0][i][1] loss.append(-torch.log( scores[0][x][y].exp() )) # check batch size == 1 ?

2. Which of the following are you using for the matching process: Chamfer Distance (CD) and Earth Mover's Distance (EMD) China? Is a gt mapped for each pred? Could you please give a suggestion on that? Thanks.

[GuoxingYan]

During training, what is the factor in Equation 12 equal to?

[GuoxingYan]

When calculating the match loss, is the method of simply assigning a nearest predictor to each label?

[hehongjie]

Hello Guoxing, do you know how to construct the permutation matrixs for model training? It seems we need to first decide the order of vertices and then made the matrix, right? But how to decide the order?

[phamkhactu]

@zorzi-s @GuoxingYan @hehongjie I saw that in prediction.py in line 42, 46,53

model.train()

It means that model is at train model not eval?? is it right? If eval must change mode to eval? If training how to train it? Many thank

[phamkhactu]

@hehongjie when I set all model to eval(), the result very poor!!

Mean IoU:  0.0008893926959595389
Mean C-IoU:  0.0007304682105452664

if want to evaluate model must train to eval model, if use train model, model can update, so not truth, this is my limited knowledge. if wrong, many sorry for you.

[hehongjie]

@phamkhactu Yes, you are right. You may refer to this answer.https://stackoverflow.com/questions/51433378/what-does-model-train-do-in-pytorch

[phamkhactu]

@phamkhactu Yes, you are right. You may refer to this answer.https://stackoverflow.com/questions/51433378/what-does-model-train-do-in-pytorch

@hehongjie as you can see that, the output validation from eval and train model very diff, but if i use train model model can update weight, can it cause wrong prediction?? I have another question that: with your design model public from your research team, the model can learn to classifier direction?

Like image, I expected that 2 polygon are diff, Does the model R2U_Net learn this?

[zorzi-s]

@zorzi-s @GuoxingYan @hehongjie I saw that in prediction.py in line 42, 46,53

model.train()

It means that model is at train model not eval?? is it right? If eval must change mode to eval? If training how to train it? Many thank

@phamkhactu @hehongjie Thank you for the interest in our method! We use model.train() during inference in order to force the batch normalization layers to use batch statistics instead of the mean and variance estimated during training. The model weights are not updated during inference. Please also refer to this conversation: link.

[phamkhactu]

@zorzi-s @GuoxingYan @hehongjie I saw that in prediction.py in line 42, 46,53

model.train()

It means that model is at train model not eval?? is it right? If eval must change mode to eval? If training how to train it? Many thank

@phamkhactu @hehongjie Thank you for the interest in our method! We use model.train() during inference in order to force the batch normalization layers to use batch statistics instead of the mean and variance estimated during training. The model weights are not updated during inference. Please also refer to this conversation: link.

@zorzi-s thank you. As I mention in above question: I have another question that: with your design model public from your research team, the model can learn to classifier direction?

Like image, I expected that 2 polygon are diff, Does the model R2U_Net learn this? For you, Your model can do that?? In my opition it can, because model learn polygon points, so direction make difference feature extraction. How about you? Many Thanks

[henokyen]

Has anyone figured out how to calculate the permutation matrix during training using the Sinkhorn algorithm?

[henokyen]

@GuoxingYan have you figured out how to calculate the partial assignment during training using the Sinkhorn algorithm?

[henokyen]

@zorzi-s How can we confirm the originality of your paper if you don't provide your training script?

[ogencoglu]

Do we have the repo for the new paper, Re:PolyWorld, @zorzi-s ? https://openaccess.thecvf.com/content/ICCV2023/papers/Zorzi_RePolyWorld_-_A_Graph_Neural_Network_for_Polygonal_Scene_Parsing_ICCV_2023_paper.pdf