clovaai / CLEval

CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks
MIT License
185 stars 28 forks source link

Change aspect ratio to avg_ver/avg_hor instead of avg_hor/avg_ver #5

Closed fafa28 closed 3 years ago

fafa28 commented 4 years ago

Problem: CLEval is detecting merge, split, overlap and miss char when the submission result is exactly the same as GT.

How to reproduce: GT file (.txt) compressed and saved as gt.zip:

182,172,290,172,290,202,182,202,super
180,196,291,200,289,241,178,237,funny

Submission file (.txt) same as GT file compressed and saved as res.zip:

182,172,290,172,290,202,182,202,super
180,196,291,200,289,241,178,237,funny

Command:

python script.py -g="gt.zip" -s="res.zip" --E2E

Output:

{'Detection': {'hmean': 0.8571428571428572,
               'precision': 0.8181818181818182,
               'recall': 0.9},
 'Detection_Metadata': {'char_false_pos': 0,
                        'char_miss': 0,
                        'char_overlap': 1,
                        'num_false_pos': 0,
                        'num_merge': 1,
                        'num_split': 1},
 'EndtoEnd': {'hmean': 0.9,
              'precision': 0.9,
              'recall': 0.9,
              'recognition_score': 1.0},
 'EndtoEnd_Metadata': {'char_false_pos': 0.0,
                       'char_miss': 1.0,
                       'num_false_pos': 0,
                       'num_merge': 1,
                       'num_split': 1}}

Desired output:

{'Detection': {'hmean': 1.0, 'precision': 1.0, 'recall': 1.0},
 'Detection_Metadata': {'char_false_pos': 0,
                        'char_miss': 0,
                        'char_overlap': 0,
                        'num_false_pos': 0,
                        'num_merge': 0,
                        'num_split': 0},
 'EndtoEnd': {'hmean': 1.0,
              'precision': 1.0,
              'recall': 1.0,
              'recognition_score': 1.0},
 'EndtoEnd_Metadata': {'char_false_pos': 0.0,
                       'char_miss': 0.0,
                       'num_false_pos': 0,
                       'num_merge': 0,
                       'num_split': 0}}

Explanation: The default value for VERTICAL_ASPECT_RATIO_THRES is 2.0 and the aspect ratio is calculated based on avg_hor/avg_ver. Hence, horizontal boxes will be considered as vertical boxes. This will result in a change in their coordinates, which will influence the PCC and lead to the false detection of merge, split, overlap and miss char.

Solution: Change the aspect ratio to avg_ver/avg_hor.