search

MitraDP / Detection-of-Surface-Defects-in-Magnetic-Tile-Images

Binary semantic segmentation of the blowhole_crack and free datasets.
MIT License
13 stars 0 forks source link

Question on evaluation metrics #3

Open rrryan2016 opened 3 years ago

rrryan2016 commented 3 years ago

Thanks for your great work and kind sharing.

I am a beginner in defect detection, but intend to do some work on Magnetic Tile dataset.

I successfully reproduced your inspiring work, and get the same evaluation result as in Out[28] in SurfaceDefectDetection_MagneticTile.ipynb.

metrics_cal

but when I wanna do a comparison based on the paper, Surface defect saliency of magnetic tile, I feel so confused because there are loads of numbers. For example, in the above figure, which row (['mean','std','min', etc.]) to select for F_beta and MAE, to match the highlighted values in the picture below.

baseline_from_magnetic_tile_dataset

Could you please give me any advice or available tutorial?

Thanks in advance.

rrryan2016 commented 3 years ago

I guess there may have an error in https://github.com/MitraDP/Detection-of-Surface-Defects-in-Magnetic-Tile-Images/blob/46eec321f3be42628912febbd0fdba08206618f0/inference.py#L108, so I re-evaluate after a tiny modification and get much more rational results as below.

metrics_cal_v2

Another question is why beta_2 is set as 0.3 in https://github.com/MitraDP/Detection-of-Surface-Defects-in-Magnetic-Tile-Images/blob/46eec321f3be42628912febbd0fdba08206618f0/metrics.py#L36

Thanks in advance.