backtime92 / CRAFT-Reimplementation

CRAFT-Pyotorch:Character Region Awareness for Text Detection Reimplementation for Pytorch
464 stars 157 forks source link

Affinity Score Problem #18

Open hanish3464 opened 5 years ago

hanish3464 commented 5 years ago

Hi! @backtime92 I have found a difference of test prediction between your pretrained model, ICDAR13,17 and CRAFT. I am not sure my opinions and I was just wondering this below part.

def add_affinity(self, image, bbox_1, bbox_2): center_1, center_2 = np.mean(bbox_1, axis=0), np.mean(bbox_2, axis=0) tl = np.mean([bbox_1[0], bbox_1[1], center_1], axis=0) bl = np.mean([bbox_1[2], bbox_1[3], center_1], axis=0) tr = np.mean([bbox_2[0], bbox_2[1], center_2], axis=0) br = np.mean([bbox_2[2], bbox_2[3], center_2], axis=0)

If you create an affinity like this code, it will fit the size of the box created by the center of the four triangles. However, the affinity of the paper seems to have been created by slightly reducing the length of the left and right rather than the size of the box.

image

After examining the CRAFT model and your model for the same image, I noticed the difference that the number of segmentation chunks was different. So the affinity GT's left and right boxes are so long that the linkage did not go away in the post-processing phase, so I thought that the segmentation chunks would not divide well.

Is there any chance that this problem affects performance? I just wonder! Thank You!

backtime92 commented 5 years ago

@hanish3464 Yes,thanks for your advice, I noticed the subtle differences,but i do not know how much distance should be reduced.And the main reason is afiinity map which impact the performance.