How to apply to text segmentation?

[Whu-wxy]

Thanks for your sharing.

I believe surface loss will also play a role in text detection models and would like to try to use it in text segmentation models.

I noticed that in probs2class, you use probs.argmax (dim = 1) to get the class, but if I only have one featuremap, the part above a certain threshold is the text area, how do I use surface loss? Can I einsum the foreground distance map and the logits map directly?

If I do this, I find that if the predicted text area is smaller than the real text area, the loss does not change, because the value of the text area in dist_map is 0, will this cause the prediction result to be too small?

[HKervadec]

Hey there,

I noticed that in probs2class, you use probs.argmax (dim = 1) to get the class, but if I only have one featuremap, the part above a certain threshold is the text area, how do I use surface loss?

Few things:

• probs2class is to be used only for visualization/predictions, not for training, as argmax is non-derivable
• argmax on two classes is effectively a thresholding with t=0.5

Can I einsum the foreground distance map and the logits map directly?

The foreground distance map yes, and the probability map ; not the raw logits. So just put then through a softmax first.

If I do this, I find that if the predicted text area is smaller than the real text area, the loss does not change, because the value of the text area in dist_map is 0, will this cause the prediction result to be too small?

The distance map is supposed to be signed:

• Positive outside the object
• Negative inside the object (the text area you are interested in)

If you end-up with the distance map inside the text area to be 0, then there is some problem in your pre-processing. Per the function definition, for the 1-class binary case, you end up with (simplified):

...
posmask = seg.astype(np.bool)
negmask = ~posmask

res = distance(negmask) * negmask - (distance(posmask) - 1) * posmask
...

The -1 is just a shift to have the distance on the boundary to be 0. Depending on the application, I guess you could remove that part.

[Whu-wxy]

Thanks for your help！I think I have made it clear, I will try it in subsequent experiments.